Mercurial > dovecot > original-hg > dovecot-1.2
annotate src/lib-storage/index/maildir/maildir-sync.c @ 9658:8ba4253adc9b HEAD tip
*-login: SSL connections didn't get closed when the client got destroyed.
author | Timo Sirainen <tss@iki.fi> |
---|---|
date | Thu, 08 May 2014 16:41:29 +0300 |
parents | 00cd9aacd03c |
children |
rev | line source |
---|---|
9532
00cd9aacd03c
Updated copyright notices to include year 2010.
Timo Sirainen <tss@iki.fi>
parents:
9338
diff
changeset
|
1 /* Copyright (c) 2004-2010 Dovecot authors, see the included COPYING file */ |
1955 | 2 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
3 /* |
1955 | 4 Here's a description of how we handle Maildir synchronization and |
5 it's problems: | |
6 | |
7 We want to be as efficient as we can. The most efficient way to | |
3520 | 8 check if changes have occurred is to stat() the new/ and cur/ |
1955 | 9 directories and uidlist file - if their mtimes haven't changed, |
10 there's no changes and we don't need to do anything. | |
11 | |
12 Problem 1: Multiple changes can happen within a single second - | |
13 nothing guarantees that once we synced it, someone else didn't just | |
14 then make a modification. Such modifications wouldn't get noticed | |
3520 | 15 until a new modification occurred later. |
1955 | 16 |
17 Problem 2: Syncing cur/ directory is much more costly than syncing | |
18 new/. Moving mails from new/ to cur/ will always change mtime of | |
19 cur/ causing us to sync it as well. | |
20 | |
21 Problem 3: We may not be able to move mail from new/ to cur/ | |
22 because we're out of quota, or simply because we're accessing a | |
23 read-only mailbox. | |
24 | |
25 | |
26 MAILDIR_SYNC_SECS | |
27 ----------------- | |
28 | |
29 Several checks below use MAILDIR_SYNC_SECS, which should be maximum | |
30 clock drift between all computers accessing the maildir (eg. via | |
31 NFS), rounded up to next second. Our default is 1 second, since | |
32 everyone should be using NTP. | |
33 | |
34 Note that setting it to 0 works only if there's only one computer | |
35 accessing the maildir. It's practically impossible to make two | |
36 clocks _exactly_ synchronized. | |
37 | |
38 It might be possible to only use file server's clock by looking at | |
39 the atime field, but I don't know how well that would actually work. | |
40 | |
41 cur directory | |
42 ------------- | |
43 | |
44 We have dirty_cur_time variable which is set to cur/ directory's | |
45 mtime when it's >= time() - MAILDIR_SYNC_SECS and we _think_ we have | |
46 synchronized the directory. | |
47 | |
48 When dirty_cur_time is non-zero, we don't synchronize the cur/ | |
49 directory until | |
50 | |
51 a) cur/'s mtime changes | |
52 b) opening a mail fails with ENOENT | |
53 c) time() > dirty_cur_time + MAILDIR_SYNC_SECS | |
54 | |
55 This allows us to modify the maildir multiple times without having | |
56 to sync it at every change. The sync will eventually be done to | |
57 make sure we didn't miss any external changes. | |
58 | |
59 The dirty_cur_time is set when: | |
60 | |
61 - we change message flags | |
62 - we expunge messages | |
63 - we move mail from new/ to cur/ | |
64 - we sync cur/ directory and it's mtime is >= time() - MAILDIR_SYNC_SECS | |
65 | |
66 It's unset when we do the final syncing, ie. when mtime is | |
67 older than time() - MAILDIR_SYNC_SECS. | |
68 | |
69 new directory | |
70 ------------- | |
71 | |
72 If new/'s mtime is >= time() - MAILDIR_SYNC_SECS, always synchronize | |
73 it. dirty_cur_time-like feature might save us a few syncs, but | |
74 that might break a client which saves a mail in one connection and | |
75 tries to fetch it in another one. new/ directory is almost always | |
76 empty, so syncing it should be very fast anyway. Actually this can | |
77 still happen if we sync only new/ dir while another client is also | |
78 moving mails from it to cur/ - it takes us a while to see them. | |
79 That's pretty unlikely to happen however, and only way to fix it | |
80 would be to always synchronize cur/ after new/. | |
81 | |
82 Normally we move all mails from new/ to cur/ whenever we sync it. If | |
83 it's not possible for some reason, we mark the mail with "probably | |
84 exists in new/ directory" flag. | |
85 | |
86 If rename() still fails because of ENOSPC or EDQUOT, we still save | |
87 the flag changes in index with dirty-flag on. When moving the mail | |
88 to cur/ directory, or when we notice it's already moved there, we | |
89 apply the flag changes to the filename, rename it and remove the | |
90 dirty flag. If there's dirty flags, this should be tried every time | |
91 after expunge or when closing the mailbox. | |
92 | |
93 uidlist | |
94 ------- | |
95 | |
96 This file contains UID <-> filename mappings. It's updated only when | |
97 new mail arrives, so it may contain filenames that have already been | |
98 deleted. Updating is done by getting uidlist.lock file, writing the | |
99 whole uidlist into it and rename()ing it over the old uidlist. This | |
100 means there's no need to lock the file for reading. | |
101 | |
102 Whenever uidlist is rewritten, it's mtime must be larger than the old | |
103 one's. Use utime() before rename() if needed. Note that inode checking | |
104 wouldn't have been sufficient as inode numbers can be reused. | |
105 | |
106 This file is usually read the first time you need to know filename for | |
107 given UID. After that it's not re-read unless new mails come that we | |
108 don't know about. | |
109 | |
110 broken clients | |
111 -------------- | |
112 | |
113 Originally the middle identifier in Maildir filename was specified | |
114 only as <process id>_<delivery counter>. That however created a | |
115 problem with randomized PIDs which made it possible that the same | |
116 PID was reused within one second. | |
117 | |
118 So if within one second a mail was delivered, MUA moved it to cur/ | |
119 and another mail was delivered by a new process using same PID as | |
120 the first one, we likely ended up overwriting the first mail when | |
121 the second mail was moved over it. | |
122 | |
123 Nowadays everyone should be giving a bit more specific identifier, | |
124 for example include microseconds in it which Dovecot does. | |
125 | |
126 There's a simple way to prevent this from happening in some cases: | |
127 Don't move the mail from new/ to cur/ if it's mtime is >= time() - | |
128 MAILDIR_SYNC_SECS. The second delivery's link() call then fails | |
129 because the file is already in new/, and it will then use a | |
130 different filename. There's a few problems with this however: | |
131 | |
132 - it requires extra stat() call which is unneeded extra I/O | |
133 - another MUA might still move the mail to cur/ | |
134 - if first file's flags are modified by either Dovecot or another | |
135 MUA, it's moved to cur/ (you _could_ just do the dirty-flagging | |
136 but that'd be ugly) | |
137 | |
138 Because this is useful only for very few people and it requires | |
139 extra I/O, I decided not to implement this. It should be however | |
140 quite easy to do since we need to be able to deal with files in new/ | |
141 in any case. | |
142 | |
143 It's also possible to never accidentally overwrite a mail by using | |
144 link() + unlink() rather than rename(). This however isn't very | |
145 good idea as it introduces potential race conditions when multiple | |
146 clients are accessing the mailbox: | |
147 | |
148 Trying to move the same mail from new/ to cur/ at the same time: | |
149 | |
150 a) Client 1 uses slightly different filename than client 2, | |
151 for example one sets read-flag on but the other doesn't. | |
152 You have the same mail duplicated now. | |
153 | |
154 b) Client 3 sees the mail between Client 1's and 2's link() calls | |
155 and changes it's flag. You have the same mail duplicated now. | |
156 | |
157 And it gets worse when they're unlink()ing in cur/ directory: | |
158 | |
159 c) Client 1 changes mails's flag and client 2 changes it back | |
160 between 1's link() and unlink(). The mail is now expunged. | |
161 | |
162 d) If you try to deal with the duplicates by unlink()ing another | |
163 one of them, you might end up unlinking both of them. | |
164 | |
165 So, what should we do then if we notice a duplicate? First of all, | |
166 it might not be a duplicate at all, readdir() might have just | |
167 returned it twice because it was just renamed. What we should do is | |
168 create a completely new base name for it and rename() it to that. | |
169 If the call fails with ENOENT, it only means that it wasn't a | |
170 duplicate after all. | |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
171 */ |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
172 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
173 #include "lib.h" |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
174 #include "ioloop.h" |
3470
346a494c2feb
Moved array declaration to array-decl.h and include it in lib.h. So array.h
Timo Sirainen <tss@iki.fi>
parents:
3453
diff
changeset
|
175 #include "array.h" |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
176 #include "buffer.h" |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
177 #include "hash.h" |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
178 #include "str.h" |
8847
43a49b3d3b5e
Maildir: Another logging improvement to EACCES error.
Timo Sirainen <tss@iki.fi>
parents:
8772
diff
changeset
|
179 #include "eacces-error.h" |
5926
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
180 #include "nfs-workarounds.h" |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
181 #include "maildir-storage.h" |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
182 #include "maildir-uidlist.h" |
5899
f29b93c0519c
Moved maildir filename related functions to maildir-filename.c
Timo Sirainen <tss@iki.fi>
parents:
5850
diff
changeset
|
183 #include "maildir-filename.h" |
4848
967de900c73a
Mailbox list indexing and related changes. Currently works only with
Timo Sirainen <tss@iki.fi>
parents:
4774
diff
changeset
|
184 #include "maildir-sync.h" |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
185 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
186 #include <stdio.h> |
2050
ee1095ccfd23
Index header changes now go through transaction log. Removed the kludgy
Timo Sirainen <tss@iki.fi>
parents:
2039
diff
changeset
|
187 #include <stddef.h> |
5588
6c89106dee21
Keyword characters weren't sorted in the maildir filename.
Timo Sirainen <tss@iki.fi>
parents:
5582
diff
changeset
|
188 #include <stdlib.h> |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
189 #include <unistd.h> |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
190 #include <dirent.h> |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
191 #include <sys/stat.h> |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
192 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
193 #define MAILDIR_FILENAME_FLAG_FOUND 128 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
194 |
3435
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
195 /* When rename()ing many files from new/ to cur/, it's possible that next |
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
196 readdir() skips some files. we don't of course wish to lose them, so we |
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
197 go and rescan the new/ directory again from beginning until no files are |
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
198 left. This value is just an optimization to avoid checking the directory |
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
199 twice unneededly. usually only NFS is the problem case. 1 is the safest |
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
200 bet here, but I guess 5 will do just fine too. */ |
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
201 #define MAILDIR_RENAME_RESCAN_COUNT 5 |
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
202 |
5397
3a0964ac3a5c
comment/duplicate link handling fixes.
Timo Sirainen <tss@iki.fi>
parents:
5392
diff
changeset
|
203 /* This is mostly to avoid infinite looping when rename() destination already |
3a0964ac3a5c
comment/duplicate link handling fixes.
Timo Sirainen <tss@iki.fi>
parents:
5392
diff
changeset
|
204 exists as the hard link of the file itself. */ |
5391
0c8705aad54c
Avoid infinite looping when buggy filesystems.
Timo Sirainen <tss@iki.fi>
parents:
5390
diff
changeset
|
205 #define MAILDIR_SCAN_DIR_MAX_COUNT 5 |
0c8705aad54c
Avoid infinite looping when buggy filesystems.
Timo Sirainen <tss@iki.fi>
parents:
5390
diff
changeset
|
206 |
5443
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
207 #define DUPE_LINKS_DELETE_SECS 30 |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
208 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
209 struct maildir_sync_context { |
3279
b698ae839a18
Moved mbox/maildir-specific variables from struct index_mailbox to
Timo Sirainen <tss@iki.fi>
parents:
3203
diff
changeset
|
210 struct maildir_mailbox *mbox; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
211 const char *new_dir, *cur_dir; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
212 |
6881
b1554dea8ee8
If we're doing a fast sync, try only once to lock uidlist.
Timo Sirainen <tss@iki.fi>
parents:
6880
diff
changeset
|
213 enum mailbox_sync_flags flags; |
5368
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
214 time_t last_touch, last_notify; |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
215 |
2818
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
216 struct maildir_uidlist_sync_ctx *uidlist_sync_ctx; |
6880
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
217 struct maildir_index_sync_context *index_sync_ctx; |
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
218 |
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
219 unsigned int partial:1; |
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
220 unsigned int locked:1; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
221 }; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
222 |
5920
00c5e3cbeaf0
Moved index syncing code to its own file.
Timo Sirainen <tss@iki.fi>
parents:
5918
diff
changeset
|
223 void maildir_sync_notify(struct maildir_sync_context *ctx) |
5368
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
224 { |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
225 time_t now; |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
226 |
5390
4eeec560df01
If saving to maildir causes flag/expunge syncs, don't crash.
Timo Sirainen <tss@iki.fi>
parents:
5388
diff
changeset
|
227 if (ctx == NULL) { |
4eeec560df01
If saving to maildir causes flag/expunge syncs, don't crash.
Timo Sirainen <tss@iki.fi>
parents:
5388
diff
changeset
|
228 /* we got here from maildir-save.c. it has no |
4eeec560df01
If saving to maildir causes flag/expunge syncs, don't crash.
Timo Sirainen <tss@iki.fi>
parents:
5388
diff
changeset
|
229 maildir_sync_context, */ |
4eeec560df01
If saving to maildir causes flag/expunge syncs, don't crash.
Timo Sirainen <tss@iki.fi>
parents:
5388
diff
changeset
|
230 return; |
4eeec560df01
If saving to maildir causes flag/expunge syncs, don't crash.
Timo Sirainen <tss@iki.fi>
parents:
5388
diff
changeset
|
231 } |
4eeec560df01
If saving to maildir causes flag/expunge syncs, don't crash.
Timo Sirainen <tss@iki.fi>
parents:
5388
diff
changeset
|
232 |
5368
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
233 now = time(NULL); |
7046
e74a1d1dca07
When doing a forced sync, we're not necessarily locked. If it takes a while,
Timo Sirainen <tss@iki.fi>
parents:
6940
diff
changeset
|
234 if (now - ctx->last_touch > MAILDIR_LOCK_TOUCH_SECS && ctx->locked) { |
5368
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
235 (void)maildir_uidlist_lock_touch(ctx->mbox->uidlist); |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
236 ctx->last_touch = now; |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
237 } |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
238 if (now - ctx->last_notify > MAIL_STORAGE_STAYALIVE_SECS) { |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
239 struct mailbox *box = &ctx->mbox->ibox.box; |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
240 |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
241 if (box->storage->callbacks->notify_ok != NULL) { |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
242 box->storage->callbacks-> |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
243 notify_ok(box, "Hang in there..", |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
244 box->storage->callback_context); |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
245 } |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
246 ctx->last_notify = now; |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
247 } |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
248 } |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
249 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
250 static struct maildir_sync_context * |
6881
b1554dea8ee8
If we're doing a fast sync, try only once to lock uidlist.
Timo Sirainen <tss@iki.fi>
parents:
6880
diff
changeset
|
251 maildir_sync_context_new(struct maildir_mailbox *mbox, |
b1554dea8ee8
If we're doing a fast sync, try only once to lock uidlist.
Timo Sirainen <tss@iki.fi>
parents:
6880
diff
changeset
|
252 enum mailbox_sync_flags flags) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
253 { |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
254 struct maildir_sync_context *ctx; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
255 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
256 ctx = t_new(struct maildir_sync_context, 1); |
3279
b698ae839a18
Moved mbox/maildir-specific variables from struct index_mailbox to
Timo Sirainen <tss@iki.fi>
parents:
3203
diff
changeset
|
257 ctx->mbox = mbox; |
b698ae839a18
Moved mbox/maildir-specific variables from struct index_mailbox to
Timo Sirainen <tss@iki.fi>
parents:
3203
diff
changeset
|
258 ctx->new_dir = t_strconcat(mbox->path, "/new", NULL); |
b698ae839a18
Moved mbox/maildir-specific variables from struct index_mailbox to
Timo Sirainen <tss@iki.fi>
parents:
3203
diff
changeset
|
259 ctx->cur_dir = t_strconcat(mbox->path, "/cur", NULL); |
5368
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
260 ctx->last_touch = ioloop_time; |
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
261 ctx->last_notify = ioloop_time; |
6881
b1554dea8ee8
If we're doing a fast sync, try only once to lock uidlist.
Timo Sirainen <tss@iki.fi>
parents:
6880
diff
changeset
|
262 ctx->flags = flags; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
263 return ctx; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
264 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
265 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
266 static void maildir_sync_deinit(struct maildir_sync_context *ctx) |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
267 { |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
268 if (ctx->uidlist_sync_ctx != NULL) |
9018
0bb192fe0abd
Maildir: More fixes to uidlist handling.
Timo Sirainen <tss@iki.fi>
parents:
9017
diff
changeset
|
269 (void)maildir_uidlist_sync_deinit(&ctx->uidlist_sync_ctx, FALSE); |
4774
615b7738a62f
Saving mails could have skipped over transactions, which caused different
Timo Sirainen <tss@iki.fi>
parents:
4612
diff
changeset
|
270 if (ctx->index_sync_ctx != NULL) { |
615b7738a62f
Saving mails could have skipped over transactions, which caused different
Timo Sirainen <tss@iki.fi>
parents:
4612
diff
changeset
|
271 (void)maildir_sync_index_finish(&ctx->index_sync_ctx, |
615b7738a62f
Saving mails could have skipped over transactions, which caused different
Timo Sirainen <tss@iki.fi>
parents:
4612
diff
changeset
|
272 TRUE, FALSE); |
615b7738a62f
Saving mails could have skipped over transactions, which caused different
Timo Sirainen <tss@iki.fi>
parents:
4612
diff
changeset
|
273 } |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
274 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
275 |
4397
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
276 static int maildir_fix_duplicate(struct maildir_sync_context *ctx, |
5910 | 277 const char *dir, const char *fname2) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
278 { |
5910 | 279 const char *fname1, *path1, *path2; |
280 const char *new_fname, *new_path; | |
281 struct stat st1, st2; | |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
282 |
5910 | 283 fname1 = maildir_uidlist_sync_get_full_filename(ctx->uidlist_sync_ctx, |
284 fname2); | |
285 i_assert(fname1 != NULL); | |
4397
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
286 |
5910 | 287 path1 = t_strconcat(dir, "/", fname1, NULL); |
288 path2 = t_strconcat(dir, "/", fname2, NULL); | |
4397
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
289 |
5910 | 290 if (stat(path1, &st1) < 0 || stat(path2, &st2) < 0) { |
4397
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
291 /* most likely the files just don't exist anymore. |
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
292 don't really care about other errors much. */ |
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
293 return 0; |
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
294 } |
5910 | 295 if (st1.st_ino == st2.st_ino && |
296 CMP_DEV_T(st1.st_dev, st2.st_dev)) { | |
5397
3a0964ac3a5c
comment/duplicate link handling fixes.
Timo Sirainen <tss@iki.fi>
parents:
5392
diff
changeset
|
297 /* Files are the same. this means either a race condition |
5443
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
298 between stat() calls, or that the files were link()ed. */ |
5910 | 299 if (st1.st_nlink > 1 && st2.st_nlink == st1.st_nlink && |
300 st1.st_ctime == st2.st_ctime && | |
301 st1.st_ctime < ioloop_time - DUPE_LINKS_DELETE_SECS) { | |
5443
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
302 /* The file has hard links and it hasn't had any |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
303 changes (such as renames) for a while, so this |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
304 isn't a race condition. |
5392
9a8402768b42
If duplicate links are found, rename() one over the other to get rid of
Timo Sirainen <tss@iki.fi>
parents:
5391
diff
changeset
|
305 |
5443
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
306 rename()ing one file on top of the other would fix |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
307 this safely, except POSIX decided that rename() |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
308 doesn't work that way. So we'll have unlink() one |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
309 and hope that another process didn't just decide to |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
310 unlink() the other (uidlist lock prevents this from |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
311 happening) */ |
5910 | 312 if (unlink(path2) == 0) |
313 i_warning("Unlinked a duplicate: %s", path2); | |
314 else { | |
5443
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
315 mail_storage_set_critical( |
5910 | 316 &ctx->mbox->storage->storage, |
317 "unlink(%s) failed: %m", path2); | |
5443
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
318 } |
293bc7d1062f
If we find duplicate hard links which haven't changed for 30 secs, unlink()
Timo Sirainen <tss@iki.fi>
parents:
5397
diff
changeset
|
319 } |
4397
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
320 return 0; |
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
321 } |
5cbabd4ccd9c
Don't go fixing duplicate maildir filenames without properly checking that
Timo Sirainen <tss@iki.fi>
parents:
4238
diff
changeset
|
322 |
5904
62ceb6b2b20d
Renamed maildir_generate_tmp_filename() to maildir_filename_generate(). Also
Timo Sirainen <tss@iki.fi>
parents:
5901
diff
changeset
|
323 new_fname = maildir_filename_generate(); |
5910 | 324 new_path = t_strconcat(ctx->mbox->path, "/new/", new_fname, NULL); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
325 |
5910 | 326 if (rename(path2, new_path) == 0) |
327 i_warning("Fixed a duplicate: %s -> %s", path2, new_fname); | |
5392
9a8402768b42
If duplicate links are found, rename() one over the other to get rid of
Timo Sirainen <tss@iki.fi>
parents:
5391
diff
changeset
|
328 else if (errno != ENOENT) { |
5910 | 329 mail_storage_set_critical(&ctx->mbox->storage->storage, |
5582 | 330 "Couldn't fix a duplicate: rename(%s, %s) failed: %m", |
5910 | 331 path2, new_path); |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
332 return -1; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
333 } |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
334 return 0; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
335 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
336 |
5926
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
337 static int |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
338 maildir_stat(struct maildir_mailbox *mbox, const char *path, struct stat *st_r) |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
339 { |
6777
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
340 int i; |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
341 |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
342 for (i = 0;; i++) { |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
343 if (nfs_safe_stat(path, st_r) == 0) |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
344 return 0; |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
345 if (errno != ENOENT || i == MAILDIR_DELETE_RETRY_COUNT) |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
346 break; |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
347 |
8772
dfcb8a6a4f5f
maildir: We didn't properly handle missing control directory at startup.
Timo Sirainen <tss@iki.fi>
parents:
8590
diff
changeset
|
348 if (!maildir_set_deleted(mbox)) |
6777
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
349 return -1; |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
350 /* try again */ |
5926
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
351 } |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
352 |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
353 mail_storage_set_critical(mbox->ibox.box.storage, |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
354 "stat(%s) failed: %m", path); |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
355 return -1; |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
356 } |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
357 |
9201
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
358 static int |
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
359 maildir_scan_dir(struct maildir_sync_context *ctx, bool new_dir, bool final) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
360 { |
5459
78eaf595359c
Removed struct index_storage abstraction. It's pointless.
Timo Sirainen <tss@iki.fi>
parents:
5443
diff
changeset
|
361 struct mail_storage *storage = &ctx->mbox->storage->storage; |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
362 const char *path; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
363 DIR *dirp; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
364 string_t *src, *dest; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
365 struct dirent *dp; |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
366 struct stat st; |
3435
6f7ce690358e
If we have rename()d more than 5 files from new/ to cur/, rescan the
Timo Sirainen <tss@iki.fi>
parents:
3417
diff
changeset
|
367 enum maildir_uidlist_rec_flag flags; |
5914
ae731dbf3a6f
Sync index changes while iterating through uidlist entries. This avoids
Timo Sirainen <tss@iki.fi>
parents:
5912
diff
changeset
|
368 unsigned int i = 0, move_count = 0; |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
369 time_t now; |
3863
55df57c028d4
Added "bool" type and changed all ints that were used as booleans to bool.
Timo Sirainen <tss@iki.fi>
parents:
3821
diff
changeset
|
370 int ret = 1; |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
371 bool move_new, check_touch, dir_changed = FALSE; |
5214
4e9d345df846
When syncing huge maildirs check once in a while if we need to update
Timo Sirainen <tss@iki.fi>
parents:
5080
diff
changeset
|
372 |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
373 path = new_dir ? ctx->new_dir : ctx->cur_dir; |
6777
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
374 for (i = 0;; i++) { |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
375 dirp = opendir(path); |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
376 if (dirp != NULL) |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
377 break; |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
378 |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
379 if (errno != ENOENT || i == MAILDIR_DELETE_RETRY_COUNT) { |
8847
43a49b3d3b5e
Maildir: Another logging improvement to EACCES error.
Timo Sirainen <tss@iki.fi>
parents:
8772
diff
changeset
|
380 if (errno == EACCES) { |
43a49b3d3b5e
Maildir: Another logging improvement to EACCES error.
Timo Sirainen <tss@iki.fi>
parents:
8772
diff
changeset
|
381 mail_storage_set_critical(storage, "%s", |
43a49b3d3b5e
Maildir: Another logging improvement to EACCES error.
Timo Sirainen <tss@iki.fi>
parents:
8772
diff
changeset
|
382 eacces_error_get("opendir", path)); |
43a49b3d3b5e
Maildir: Another logging improvement to EACCES error.
Timo Sirainen <tss@iki.fi>
parents:
8772
diff
changeset
|
383 } else { |
43a49b3d3b5e
Maildir: Another logging improvement to EACCES error.
Timo Sirainen <tss@iki.fi>
parents:
8772
diff
changeset
|
384 mail_storage_set_critical(storage, |
43a49b3d3b5e
Maildir: Another logging improvement to EACCES error.
Timo Sirainen <tss@iki.fi>
parents:
8772
diff
changeset
|
385 "opendir(%s) failed: %m", path); |
43a49b3d3b5e
Maildir: Another logging improvement to EACCES error.
Timo Sirainen <tss@iki.fi>
parents:
8772
diff
changeset
|
386 } |
5926
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
387 return -1; |
095b3adc537b
Handle mailbox deletions while syncing silently.
Timo Sirainen <tss@iki.fi>
parents:
5920
diff
changeset
|
388 } |
6777
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
389 |
8772
dfcb8a6a4f5f
maildir: We didn't properly handle missing control directory at startup.
Timo Sirainen <tss@iki.fi>
parents:
8590
diff
changeset
|
390 if (!maildir_set_deleted(ctx->mbox)) |
6777
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
391 return -1; |
186b164a9579
Delay creating directories until we really need them.
Timo Sirainen <tss@iki.fi>
parents:
6771
diff
changeset
|
392 /* try again */ |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
393 } |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
394 |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
395 #ifdef HAVE_DIRFD |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
396 if (fstat(dirfd(dirp), &st) < 0) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
397 mail_storage_set_critical(storage, |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
398 "fstat(%s) failed: %m", path); |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
399 (void)closedir(dirp); |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
400 return -1; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
401 } |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
402 #else |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
403 if (maildir_stat(ctx->mbox, path, &st) < 0) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
404 (void)closedir(dirp); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
405 return -1; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
406 } |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
407 #endif |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
408 |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
409 now = time(NULL); |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
410 if (new_dir) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
411 ctx->mbox->maildir_hdr.new_check_time = now; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
412 ctx->mbox->maildir_hdr.new_mtime = st.st_mtime; |
7526
a957567706ec
Replaced HAVE_ST_NSEC macro checks with more generic ST_?TIME_NSEC() macros
Timo Sirainen <tss@iki.fi>
parents:
7515
diff
changeset
|
413 ctx->mbox->maildir_hdr.new_mtime_nsecs = ST_MTIME_NSEC(st); |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
414 } else { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
415 ctx->mbox->maildir_hdr.cur_check_time = now; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
416 ctx->mbox->maildir_hdr.cur_mtime = st.st_mtime; |
7526
a957567706ec
Replaced HAVE_ST_NSEC macro checks with more generic ST_?TIME_NSEC() macros
Timo Sirainen <tss@iki.fi>
parents:
7515
diff
changeset
|
417 ctx->mbox->maildir_hdr.cur_mtime_nsecs = ST_MTIME_NSEC(st); |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
418 } |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
419 |
2121 | 420 src = t_str_new(1024); |
421 dest = t_str_new(1024); | |
422 | |
3279
b698ae839a18
Moved mbox/maildir-specific variables from struct index_mailbox to
Timo Sirainen <tss@iki.fi>
parents:
3203
diff
changeset
|
423 move_new = new_dir && !mailbox_is_readonly(&ctx->mbox->ibox.box) && |
6880
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
424 !ctx->mbox->ibox.keep_recent && ctx->locked; |
5911
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
425 |
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
426 errno = 0; |
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
427 for (; (dp = readdir(dirp)) != NULL; errno = 0) { |
2258
087a43e29492
No maildir filename checking after all.
Timo Sirainen <tss@iki.fi>
parents:
2256
diff
changeset
|
428 if (dp->d_name[0] == '.') |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
429 continue; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
430 |
5214
4e9d345df846
When syncing huge maildirs check once in a while if we need to update
Timo Sirainen <tss@iki.fi>
parents:
5080
diff
changeset
|
431 check_touch = FALSE; |
1947 | 432 flags = 0; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
433 if (move_new) { |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
434 str_truncate(src, 0); |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
435 str_truncate(dest, 0); |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
436 str_printfa(src, "%s/%s", ctx->new_dir, dp->d_name); |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
437 str_printfa(dest, "%s/%s", ctx->cur_dir, dp->d_name); |
3417
b0bdf32564b7
Replaced ':' and ',' character usages with #defines, so they can be changed
Timo Sirainen <tss@iki.fi>
parents:
3340
diff
changeset
|
438 if (strchr(dp->d_name, MAILDIR_INFO_SEP) == NULL) { |
b0bdf32564b7
Replaced ':' and ',' character usages with #defines, so they can be changed
Timo Sirainen <tss@iki.fi>
parents:
3340
diff
changeset
|
439 str_append(dest, MAILDIR_FLAGS_FULL_SEP); |
b0bdf32564b7
Replaced ':' and ',' character usages with #defines, so they can be changed
Timo Sirainen <tss@iki.fi>
parents:
3340
diff
changeset
|
440 } |
1947 | 441 if (rename(str_c(src), str_c(dest)) == 0) { |
1984 | 442 /* we moved it - it's \Recent for us */ |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
443 dir_changed = TRUE; |
5914
ae731dbf3a6f
Sync index changes while iterating through uidlist entries. This avoids
Timo Sirainen <tss@iki.fi>
parents:
5912
diff
changeset
|
444 move_count++; |
1947 | 445 flags |= MAILDIR_UIDLIST_REC_FLAG_MOVED | |
446 MAILDIR_UIDLIST_REC_FLAG_RECENT; | |
447 } else if (ENOTFOUND(errno)) { | |
448 /* someone else moved it already */ | |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
449 dir_changed = TRUE; |
5914
ae731dbf3a6f
Sync index changes while iterating through uidlist entries. This avoids
Timo Sirainen <tss@iki.fi>
parents:
5912
diff
changeset
|
450 move_count++; |
6037
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
451 flags |= MAILDIR_UIDLIST_REC_FLAG_MOVED | |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
452 MAILDIR_UIDLIST_REC_FLAG_RECENT; |
4199
fa135b615b01
If maildir is readonly, don't complain about rename() failing to move mails
Timo Sirainen <tss@iki.fi>
parents:
4152
diff
changeset
|
453 } else if (ENOSPACE(errno) || errno == EACCES) { |
fa135b615b01
If maildir is readonly, don't complain about rename() failing to move mails
Timo Sirainen <tss@iki.fi>
parents:
4152
diff
changeset
|
454 /* not enough disk space / read-only maildir, |
fa135b615b01
If maildir is readonly, don't complain about rename() failing to move mails
Timo Sirainen <tss@iki.fi>
parents:
4152
diff
changeset
|
455 leave here */ |
6037
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
456 flags |= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
457 move_new = FALSE; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
458 } else { |
6037
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
459 flags |= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
460 mail_storage_set_critical(storage, |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
461 "rename(%s, %s) failed: %m", |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
462 str_c(src), str_c(dest)); |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
463 } |
5914
ae731dbf3a6f
Sync index changes while iterating through uidlist entries. This avoids
Timo Sirainen <tss@iki.fi>
parents:
5912
diff
changeset
|
464 if ((move_count % MAILDIR_SLOW_MOVE_COUNT) == 0) |
ae731dbf3a6f
Sync index changes while iterating through uidlist entries. This avoids
Timo Sirainen <tss@iki.fi>
parents:
5912
diff
changeset
|
465 maildir_sync_notify(ctx); |
1947 | 466 } else if (new_dir) { |
467 flags |= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR | | |
468 MAILDIR_UIDLIST_REC_FLAG_RECENT; | |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
469 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
470 |
5914
ae731dbf3a6f
Sync index changes while iterating through uidlist entries. This avoids
Timo Sirainen <tss@iki.fi>
parents:
5912
diff
changeset
|
471 i++; |
ae731dbf3a6f
Sync index changes while iterating through uidlist entries. This avoids
Timo Sirainen <tss@iki.fi>
parents:
5912
diff
changeset
|
472 if ((i % MAILDIR_SLOW_CHECK_COUNT) == 0) |
ae731dbf3a6f
Sync index changes while iterating through uidlist entries. This avoids
Timo Sirainen <tss@iki.fi>
parents:
5912
diff
changeset
|
473 maildir_sync_notify(ctx); |
5214
4e9d345df846
When syncing huge maildirs check once in a while if we need to update
Timo Sirainen <tss@iki.fi>
parents:
5080
diff
changeset
|
474 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
475 ret = maildir_uidlist_sync_next(ctx->uidlist_sync_ctx, |
1947 | 476 dp->d_name, flags); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
477 if (ret <= 0) { |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
478 if (ret < 0) |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
479 break; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
480 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
481 /* possibly duplicate - try fixing it */ |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
482 T_BEGIN { |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
483 ret = maildir_fix_duplicate(ctx, path, |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
484 dp->d_name); |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
485 } T_END; |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
486 if (ret < 0) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
487 break; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
488 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
489 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
490 |
9201
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
491 #ifdef __APPLE__ |
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
492 if (errno == EINVAL && move_count > 0 && !final) { |
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
493 /* OS X HFS+: readdir() fails sometimes when rename() |
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
494 have been done. */ |
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
495 move_count = MAILDIR_RENAME_RESCAN_COUNT + 1; |
9202
797474444551
OSX+Maildir: Also don't log an error if readdir() fails with EINVAL.
Timo Sirainen <tss@iki.fi>
parents:
9201
diff
changeset
|
496 } else |
9201
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
497 #endif |
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
498 |
5911
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
499 if (errno != 0) { |
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
500 mail_storage_set_critical(storage, |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
501 "readdir(%s) failed: %m", path); |
5911
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
502 ret = -1; |
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
503 } |
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
504 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
505 if (closedir(dirp) < 0) { |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
506 mail_storage_set_critical(storage, |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
507 "closedir(%s) failed: %m", path); |
5911
8bb5c7a69405
If readdir() or closedir() fails, return failure.
Timo Sirainen <tss@iki.fi>
parents:
5910
diff
changeset
|
508 ret = -1; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
509 } |
2121 | 510 |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
511 if (dir_changed) { |
8876
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
512 /* save the exact new times. the new mtimes should be >= |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
513 "now", but just in case something weird happens and mtime |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
514 doesn't update, use "now". */ |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
515 if (stat(ctx->new_dir, &st) == 0) { |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
516 ctx->mbox->maildir_hdr.new_check_time = |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
517 I_MAX(st.st_mtime, now); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
518 ctx->mbox->maildir_hdr.new_mtime = st.st_mtime; |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
519 ctx->mbox->maildir_hdr.new_mtime_nsecs = |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
520 ST_MTIME_NSEC(st); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
521 } |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
522 if (stat(ctx->cur_dir, &st) == 0) { |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
523 ctx->mbox->maildir_hdr.new_check_time = |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
524 I_MAX(st.st_mtime, now); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
525 ctx->mbox->maildir_hdr.cur_mtime = st.st_mtime; |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
526 ctx->mbox->maildir_hdr.cur_mtime_nsecs = |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
527 ST_MTIME_NSEC(st); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
528 } |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
529 } |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
530 |
5368
7d45edb81fe4
When copying/syncing a lot of mails, send "* OK Hang in there" replies to
Timo Sirainen <tss@iki.fi>
parents:
5307
diff
changeset
|
531 return ret < 0 ? -1 : |
9201
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
532 (move_count <= MAILDIR_RENAME_RESCAN_COUNT || final ? 0 : 1); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
533 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
534 |
7515
c14a2b0a3126
Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date
Timo Sirainen <tss@iki.fi>
parents:
7506
diff
changeset
|
535 int maildir_sync_header_refresh(struct maildir_mailbox *mbox) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
536 { |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
537 const void *data; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
538 size_t data_size; |
4848
967de900c73a
Mailbox list indexing and related changes. Currently works only with
Timo Sirainen <tss@iki.fi>
parents:
4774
diff
changeset
|
539 |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
540 if (mail_index_refresh(mbox->ibox.index) < 0) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
541 mail_storage_set_index_error(&mbox->ibox); |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
542 return -1; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
543 } |
3472
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
544 |
6275
913b188f4dd4
Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents:
6037
diff
changeset
|
545 mail_index_get_header_ext(mbox->ibox.view, mbox->maildir_ext_id, |
913b188f4dd4
Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents:
6037
diff
changeset
|
546 &data, &data_size); |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
547 if (data_size == 0) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
548 /* doesn't exist */ |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
549 return 0; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
550 } |
4848
967de900c73a
Mailbox list indexing and related changes. Currently works only with
Timo Sirainen <tss@iki.fi>
parents:
4774
diff
changeset
|
551 |
7515
c14a2b0a3126
Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date
Timo Sirainen <tss@iki.fi>
parents:
7506
diff
changeset
|
552 memcpy(&mbox->maildir_hdr, data, |
c14a2b0a3126
Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date
Timo Sirainen <tss@iki.fi>
parents:
7506
diff
changeset
|
553 I_MIN(sizeof(mbox->maildir_hdr), data_size)); |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
554 return 0; |
3472
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
555 } |
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
556 |
7066
048325df1ec4
If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands),
Timo Sirainen <tss@iki.fi>
parents:
7046
diff
changeset
|
557 static int maildir_sync_quick_check(struct maildir_mailbox *mbox, bool undirty, |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
558 const char *new_dir, const char *cur_dir, |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
559 bool *new_changed_r, bool *cur_changed_r) |
3472
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
560 { |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
561 #define DIR_DELAYED_REFRESH(hdr, name) \ |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
562 ((hdr)->name ## _check_time <= \ |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
563 (hdr)->name ## _mtime + MAILDIR_SYNC_SECS && \ |
7066
048325df1ec4
If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands),
Timo Sirainen <tss@iki.fi>
parents:
7046
diff
changeset
|
564 (undirty || \ |
048325df1ec4
If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands),
Timo Sirainen <tss@iki.fi>
parents:
7046
diff
changeset
|
565 (time_t)(hdr)->name ## _check_time < ioloop_time - MAILDIR_SYNC_SECS)) |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
566 |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
567 #define DIR_MTIME_CHANGED(st, hdr, name) \ |
5983 | 568 ((st).st_mtime != (time_t)(hdr)->name ## _mtime || \ |
7526
a957567706ec
Replaced HAVE_ST_NSEC macro checks with more generic ST_?TIME_NSEC() macros
Timo Sirainen <tss@iki.fi>
parents:
7515
diff
changeset
|
569 !ST_NTIMES_EQUAL(ST_MTIME_NSEC(st), (hdr)->name ## _mtime_nsecs)) |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
570 |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
571 struct maildir_index_header *hdr = &mbox->maildir_hdr; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
572 struct stat new_st, cur_st; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
573 bool refreshed = FALSE, check_new = FALSE, check_cur = FALSE; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
574 |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
575 if (mbox->maildir_hdr.new_mtime == 0) { |
7515
c14a2b0a3126
Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date
Timo Sirainen <tss@iki.fi>
parents:
7506
diff
changeset
|
576 if (maildir_sync_header_refresh(mbox) < 0) |
6328 | 577 return -1; |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
578 if (mbox->maildir_hdr.new_mtime == 0) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
579 /* first sync */ |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
580 *new_changed_r = *cur_changed_r = TRUE; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
581 return 0; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
582 } |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
583 } |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
584 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
585 *new_changed_r = *cur_changed_r = FALSE; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
586 |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
587 /* try to avoid stat()ing by first checking delayed changes */ |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
588 if (DIR_DELAYED_REFRESH(hdr, new) || |
8876
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
589 (DIR_DELAYED_REFRESH(hdr, cur) && !mbox->very_dirty_syncs)) { |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
590 /* refresh index and try again */ |
7515
c14a2b0a3126
Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date
Timo Sirainen <tss@iki.fi>
parents:
7506
diff
changeset
|
591 if (maildir_sync_header_refresh(mbox) < 0) |
6328 | 592 return -1; |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
593 refreshed = TRUE; |
3472
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
594 |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
595 if (DIR_DELAYED_REFRESH(hdr, new)) |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
596 *new_changed_r = TRUE; |
8876
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
597 if (DIR_DELAYED_REFRESH(hdr, cur) && !mbox->very_dirty_syncs) |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
598 *cur_changed_r = TRUE; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
599 if (*new_changed_r && *cur_changed_r) |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
600 return 0; |
1954 | 601 } |
602 | |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
603 if (!*new_changed_r) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
604 if (maildir_stat(mbox, new_dir, &new_st) < 0) |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
605 return -1; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
606 check_new = TRUE; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
607 } |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
608 if (!*cur_changed_r) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
609 if (maildir_stat(mbox, cur_dir, &cur_st) < 0) |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
610 return -1; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
611 check_cur = TRUE; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
612 } |
1954 | 613 |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
614 for (;;) { |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
615 if (check_new) |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
616 *new_changed_r = DIR_MTIME_CHANGED(new_st, hdr, new); |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
617 if (check_cur) |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
618 *cur_changed_r = DIR_MTIME_CHANGED(cur_st, hdr, cur); |
1984 | 619 |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
620 if ((!*new_changed_r && !*cur_changed_r) || refreshed) |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
621 break; |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
622 |
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
623 /* refresh index and try again */ |
7515
c14a2b0a3126
Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date
Timo Sirainen <tss@iki.fi>
parents:
7506
diff
changeset
|
624 if (maildir_sync_header_refresh(mbox) < 0) |
6328 | 625 return -1; |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
626 refreshed = TRUE; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
627 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
628 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
629 return 0; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
630 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
631 |
6027
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
632 static void maildir_sync_update_next_uid(struct maildir_mailbox *mbox) |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
633 { |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
634 const struct mail_index_header *hdr; |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
635 uint32_t uid_validity, next_uid; |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
636 |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
637 hdr = mail_index_get_header(mbox->ibox.view); |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
638 if (hdr->uid_validity == 0) |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
639 return; |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
640 |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
641 uid_validity = maildir_uidlist_get_uid_validity(mbox->uidlist); |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
642 next_uid = maildir_uidlist_get_next_uid(mbox->uidlist); |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
643 |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
644 if (uid_validity == hdr->uid_validity || uid_validity == 0) { |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
645 /* make sure uidlist's next_uid is at least as large as |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
646 index file's. typically this happens only if uidlist gets |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
647 deleted. */ |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
648 maildir_uidlist_set_uid_validity(mbox->uidlist, |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
649 hdr->uid_validity); |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
650 maildir_uidlist_set_next_uid(mbox->uidlist, |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
651 hdr->next_uid, FALSE); |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
652 } |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
653 } |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
654 |
9067
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
655 static bool have_recent_messages(struct maildir_sync_context *ctx) |
6037
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
656 { |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
657 const struct mail_index_header *hdr; |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
658 |
6859
64e0f45e8dc3
Delayed NFS attribute cache flushing had some bug, so removed it for now.
Timo Sirainen <tss@iki.fi>
parents:
6800
diff
changeset
|
659 (void)maildir_uidlist_refresh(ctx->mbox->uidlist); |
6037
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
660 |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
661 /* if there are files in new/, we'll need to move them. we'll check |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
662 this by checking if we have any recent messages */ |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
663 hdr = mail_index_get_header(ctx->mbox->ibox.view); |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
664 return hdr->first_recent_uid < |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
665 maildir_uidlist_get_next_uid(ctx->mbox->uidlist); |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
666 } |
d911d943438e
Recent flag handling rewrite. Still not perfect with maildir.
Timo Sirainen <tss@iki.fi>
parents:
6027
diff
changeset
|
667 |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
668 static int maildir_sync_get_changes(struct maildir_sync_context *ctx, |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
669 bool *new_changed_r, bool *cur_changed_r) |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
670 { |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
671 enum mail_index_sync_flags flags = 0; |
7066
048325df1ec4
If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands),
Timo Sirainen <tss@iki.fi>
parents:
7046
diff
changeset
|
672 bool undirty = (ctx->flags & MAILBOX_SYNC_FLAG_FULL_READ) != 0; |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
673 |
7066
048325df1ec4
If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands),
Timo Sirainen <tss@iki.fi>
parents:
7046
diff
changeset
|
674 if (maildir_sync_quick_check(ctx->mbox, undirty, |
048325df1ec4
If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands),
Timo Sirainen <tss@iki.fi>
parents:
7046
diff
changeset
|
675 ctx->new_dir, ctx->cur_dir, |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
676 new_changed_r, cur_changed_r) < 0) |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
677 return -1; |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
678 |
9067
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
679 if (have_recent_messages(ctx)) { |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
680 if (!ctx->mbox->ibox.keep_recent) { |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
681 *new_changed_r = TRUE; |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
682 } else if (*new_changed_r) { |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
683 /* we have some recent messages and new/ has changed. |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
684 if messages had been externally deleted from new/, |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
685 we need to get them out of index. this requires that |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
686 we make sure they weren't just moved to cur/. */ |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
687 *cur_changed_r = TRUE; |
e7792f4509e6
Maildir: Make sure messages are removed from index if they are deleted externally from new/.
Timo Sirainen <tss@iki.fi>
parents:
9018
diff
changeset
|
688 } |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
689 } |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
690 |
9338
e1685886c795
maildir: Fixed maildir_sync_get_changes() to work as it should have.
Timo Sirainen <tss@iki.fi>
parents:
9202
diff
changeset
|
691 if (*new_changed_r || *cur_changed_r) |
e1685886c795
maildir: Fixed maildir_sync_get_changes() to work as it should have.
Timo Sirainen <tss@iki.fi>
parents:
9202
diff
changeset
|
692 return 1; |
e1685886c795
maildir: Fixed maildir_sync_get_changes() to work as it should have.
Timo Sirainen <tss@iki.fi>
parents:
9202
diff
changeset
|
693 |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
694 if (!ctx->mbox->ibox.keep_recent) |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
695 flags |= MAIL_INDEX_SYNC_FLAG_DROP_RECENT; |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
696 |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
697 return mail_index_sync_have_any(ctx->mbox->ibox.index, flags) ? 1 : 0; |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
698 } |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
699 |
3863
55df57c028d4
Added "bool" type and changed all ints that were used as booleans to bool.
Timo Sirainen <tss@iki.fi>
parents:
3821
diff
changeset
|
700 static int maildir_sync_context(struct maildir_sync_context *ctx, bool forced, |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
701 uint32_t *find_uid, bool *lost_files_r) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
702 { |
6878
0c3ab0aef33c
maildir_uidlist_sync_init() was called wrong, although it just happened to
Timo Sirainen <tss@iki.fi>
parents:
6859
diff
changeset
|
703 enum maildir_uidlist_sync_flags sync_flags; |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
704 enum maildir_uidlist_rec_flag flags; |
8554
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
705 bool new_changed, cur_changed, lock_failure; |
9017
0aa17f3e4a6d
Maildir: Handle uidlist errors better.
Timo Sirainen <tss@iki.fi>
parents:
8997
diff
changeset
|
706 const char *fname; |
3863
55df57c028d4
Added "bool" type and changed all ints that were used as booleans to bool.
Timo Sirainen <tss@iki.fi>
parents:
3821
diff
changeset
|
707 int ret; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
708 |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
709 *lost_files_r = FALSE; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
710 |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
711 if (forced) |
2140
e2cd51b99359
"readonly sync" -> "forced sync"
Timo Sirainen <tss@iki.fi>
parents:
2123
diff
changeset
|
712 new_changed = cur_changed = TRUE; |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
713 else { |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
714 ret = maildir_sync_get_changes(ctx, &new_changed, &cur_changed); |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
715 if (ret <= 0) |
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
716 return ret; |
2140
e2cd51b99359
"readonly sync" -> "forced sync"
Timo Sirainen <tss@iki.fi>
parents:
2123
diff
changeset
|
717 } |
1947 | 718 |
2818
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
719 /* |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
720 Locking, locking, locking.. Wasn't maildir supposed to be lockless? |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
721 |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
722 We can get here either as beginning a real maildir sync, or when |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
723 committing changes to maildir but a file was lost (maybe renamed). |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
724 |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
725 So, we're going to need two locks. One for index and one for |
4238
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
726 uidlist. To avoid deadlocking do the uidlist lock always first. |
2121 | 727 |
2818
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
728 uidlist is needed only for figuring out UIDs for newly seen files, |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
729 so theoretically we wouldn't need to lock it unless there are new |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
730 files. It has a few problems though, assuming the index lock didn't |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
731 already protect it (eg. in-memory indexes): |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
732 |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
733 1. Just because you see a new file which doesn't exist in uidlist |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
734 file, doesn't mean that the file really exists anymore, or that |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
735 your readdir() lists all new files. Meaning that this is possible: |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
736 |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
737 A: opendir(), readdir() -> new file ... |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
738 -- new files are written to the maildir -- |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
739 B: opendir(), readdir() -> new file, lock uidlist, |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
740 readdir() -> another new file, rewrite uidlist, unlock |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
741 A: ... lock uidlist, readdir() -> nothing left, rewrite uidlist, |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
742 unlock |
2173
9438951e243f
Don't crash if we're syncing last commit to maildir, but some file was lost
Timo Sirainen <tss@iki.fi>
parents:
2140
diff
changeset
|
743 |
2818
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
744 The second time running A didn't see the two new files. To handle |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
745 this correctly, it must not remove the new unseen files from |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
746 uidlist. This is possible to do, but adds extra complexity. |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
747 |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
748 2. If another process is rename()ing files while we are |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
749 readdir()ing, it's possible that readdir() never lists some files, |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
750 causing Dovecot to assume they were expunged. In next sync they |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
751 would show up again, but client could have already been notified of |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
752 that and they would show up under new UIDs, so the damage is |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
753 already done. |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
754 |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
755 Both of the problems can be avoided if we simply lock the uidlist |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
756 before syncing and keep it until sync is finished. Typically this |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
757 would happen in any case, as there is the index lock.. |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
758 |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
759 The second case is still a problem with external changes though, |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
760 because maildir doesn't require any kind of locking. Luckily this |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
761 problem rarely happens except under high amount of modifications. |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
762 */ |
a758a5b542bb
Always protect maildir syncing with uidlist lock. Before we only tried to
Timo Sirainen <tss@iki.fi>
parents:
2816
diff
changeset
|
763 |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
764 if (!cur_changed) { |
6880
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
765 ctx->partial = TRUE; |
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
766 sync_flags = MAILDIR_UIDLIST_SYNC_PARTIAL; |
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
767 } else { |
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
768 ctx->partial = FALSE; |
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
769 sync_flags = 0; |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
770 if (forced) |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
771 sync_flags |= MAILDIR_UIDLIST_SYNC_FORCE; |
6881
b1554dea8ee8
If we're doing a fast sync, try only once to lock uidlist.
Timo Sirainen <tss@iki.fi>
parents:
6880
diff
changeset
|
772 if ((ctx->flags & MAILBOX_SYNC_FLAG_FAST) != 0) |
b1554dea8ee8
If we're doing a fast sync, try only once to lock uidlist.
Timo Sirainen <tss@iki.fi>
parents:
6880
diff
changeset
|
773 sync_flags |= MAILDIR_UIDLIST_SYNC_TRYLOCK; |
6880
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
774 } |
6878
0c3ab0aef33c
maildir_uidlist_sync_init() was called wrong, although it just happened to
Timo Sirainen <tss@iki.fi>
parents:
6859
diff
changeset
|
775 ret = maildir_uidlist_sync_init(ctx->mbox->uidlist, sync_flags, |
3530
e9695ec7925b
Recursive maildir uidlist syncs caused assert crashes. Also did some
Timo Sirainen <tss@iki.fi>
parents:
3520
diff
changeset
|
776 &ctx->uidlist_sync_ctx); |
8554
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
777 lock_failure = ret <= 0; |
3530
e9695ec7925b
Recursive maildir uidlist syncs caused assert crashes. Also did some
Timo Sirainen <tss@iki.fi>
parents:
3520
diff
changeset
|
778 if (ret <= 0) { |
8554
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
779 struct mail_storage *storage = ctx->mbox->ibox.box.storage; |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
780 |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
781 if (ret == 0) { |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
782 /* timeout */ |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
783 return 0; |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
784 } |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
785 /* locking failed. sync anyway without locking so that it's |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
786 possible to expunge messages when out of quota. */ |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
787 if (forced) { |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
788 /* we're already forcing a sync, we're trying to find |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
789 a message that was probably already expunged, don't |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
790 loop for a long time trying to find it. */ |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
791 return -1; |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
792 } |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
793 ret = maildir_uidlist_sync_init(ctx->mbox->uidlist, sync_flags | |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
794 MAILDIR_UIDLIST_SYNC_NOLOCK, |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
795 &ctx->uidlist_sync_ctx); |
8997
cceea391d04d
maildir: Don't assert-crash if we have no read-access to the Maildir.
Timo Sirainen <tss@iki.fi>
parents:
8876
diff
changeset
|
796 if (ret <= 0) { |
cceea391d04d
maildir: Don't assert-crash if we have no read-access to the Maildir.
Timo Sirainen <tss@iki.fi>
parents:
8876
diff
changeset
|
797 i_assert(ret != 0); |
cceea391d04d
maildir: Don't assert-crash if we have no read-access to the Maildir.
Timo Sirainen <tss@iki.fi>
parents:
8876
diff
changeset
|
798 return -1; |
cceea391d04d
maildir: Don't assert-crash if we have no read-access to the Maildir.
Timo Sirainen <tss@iki.fi>
parents:
8876
diff
changeset
|
799 } |
8554
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
800 |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
801 if (storage->callbacks->notify_no != NULL) { |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
802 storage->callbacks->notify_no(&ctx->mbox->ibox.box, |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
803 "Internal mailbox synchronization failure, " |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
804 "showing only old mails.", |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
805 storage->callback_context); |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
806 } |
2140
e2cd51b99359
"readonly sync" -> "forced sync"
Timo Sirainen <tss@iki.fi>
parents:
2123
diff
changeset
|
807 } |
6880
724c8f12eed2
maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try
Timo Sirainen <tss@iki.fi>
parents:
6878
diff
changeset
|
808 ctx->locked = maildir_uidlist_is_locked(ctx->mbox->uidlist); |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
809 if (!ctx->locked) |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
810 ctx->partial = TRUE; |
2123 | 811 |
8554
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
812 if (!ctx->mbox->syncing_commit && (ctx->locked || lock_failure)) { |
5920
00c5e3cbeaf0
Moved index syncing code to its own file.
Timo Sirainen <tss@iki.fi>
parents:
5918
diff
changeset
|
813 if (maildir_sync_index_begin(ctx->mbox, ctx, |
4238
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
814 &ctx->index_sync_ctx) < 0) |
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
815 return -1; |
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
816 } |
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
817 |
3447
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
818 if (new_changed || cur_changed) { |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
819 /* if we're going to check cur/ dir our current logic requires |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
820 that new/ dir is checked as well. it's a good idea anyway. */ |
5391
0c8705aad54c
Avoid infinite looping when buggy filesystems.
Timo Sirainen <tss@iki.fi>
parents:
5390
diff
changeset
|
821 unsigned int count = 0; |
9201
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
822 bool final = FALSE; |
5391
0c8705aad54c
Avoid infinite looping when buggy filesystems.
Timo Sirainen <tss@iki.fi>
parents:
5390
diff
changeset
|
823 |
9201
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
824 while ((ret = maildir_scan_dir(ctx, TRUE, final)) > 0) { |
3447
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
825 /* rename()d at least some files, which might have |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
826 caused some other files to be missed. check again |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
827 (see MAILDIR_RENAME_RESCAN_COUNT). */ |
9201
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
828 if (++count >= MAILDIR_SCAN_DIR_MAX_COUNT) |
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
829 final = TRUE; |
3447
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
830 } |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
831 if (ret < 0) |
1954 | 832 return -1; |
3447
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
833 |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
834 if (cur_changed) { |
9201
4e74f4651f5b
OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+.
Timo Sirainen <tss@iki.fi>
parents:
9067
diff
changeset
|
835 if (maildir_scan_dir(ctx, FALSE, TRUE) < 0) |
3447
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
836 return -1; |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
837 } |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
838 |
6027
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
839 maildir_sync_update_next_uid(ctx->mbox); |
3b260c6d9207
If uidlist gets deleted but indexes not, give UIDs to messages beginning
Timo Sirainen <tss@iki.fi>
parents:
5983
diff
changeset
|
840 |
3447
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
841 /* finish uidlist syncing, but keep it still locked */ |
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
842 maildir_uidlist_sync_finish(ctx->uidlist_sync_ctx); |
1954 | 843 } |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
844 |
6883
15df8bb27d7a
If we did unlocked maildir sync, make sure we sync it later with locking.
Timo Sirainen <tss@iki.fi>
parents:
6881
diff
changeset
|
845 if (!ctx->locked) { |
15df8bb27d7a
If we did unlocked maildir sync, make sure we sync it later with locking.
Timo Sirainen <tss@iki.fi>
parents:
6881
diff
changeset
|
846 /* make sure we sync the maildir later */ |
15df8bb27d7a
If we did unlocked maildir sync, make sure we sync it later with locking.
Timo Sirainen <tss@iki.fi>
parents:
6881
diff
changeset
|
847 ctx->mbox->maildir_hdr.new_mtime = 0; |
15df8bb27d7a
If we did unlocked maildir sync, make sure we sync it later with locking.
Timo Sirainen <tss@iki.fi>
parents:
6881
diff
changeset
|
848 ctx->mbox->maildir_hdr.cur_mtime = 0; |
15df8bb27d7a
If we did unlocked maildir sync, make sure we sync it later with locking.
Timo Sirainen <tss@iki.fi>
parents:
6881
diff
changeset
|
849 } |
15df8bb27d7a
If we did unlocked maildir sync, make sure we sync it later with locking.
Timo Sirainen <tss@iki.fi>
parents:
6881
diff
changeset
|
850 |
8554
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
851 if (ctx->index_sync_ctx != NULL) { |
3530
e9695ec7925b
Recursive maildir uidlist syncs caused assert crashes. Also did some
Timo Sirainen <tss@iki.fi>
parents:
3520
diff
changeset
|
852 /* NOTE: index syncing here might cause a re-sync due to |
e9695ec7925b
Recursive maildir uidlist syncs caused assert crashes. Also did some
Timo Sirainen <tss@iki.fi>
parents:
3520
diff
changeset
|
853 files getting lost, so this function might be called |
5927
b9865213da42
Store syncing information to maildir extension header instead of kludging
Timo Sirainen <tss@iki.fi>
parents:
5926
diff
changeset
|
854 re-entrantly. */ |
4238
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
855 ret = maildir_sync_index(ctx->index_sync_ctx, ctx->partial); |
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
856 if (maildir_sync_index_finish(&ctx->index_sync_ctx, |
4774
615b7738a62f
Saving mails could have skipped over transactions, which caused different
Timo Sirainen <tss@iki.fi>
parents:
4612
diff
changeset
|
857 ret < 0, FALSE) < 0) |
2173
9438951e243f
Don't crash if we're syncing last commit to maildir, but some file was lost
Timo Sirainen <tss@iki.fi>
parents:
2140
diff
changeset
|
858 return -1; |
4238
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
859 |
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
860 if (ret < 0) |
3c8b191b0019
Adding mail to index while saving it had a race condition. Fixing it
Timo Sirainen <timo.sirainen@movial.fi>
parents:
4199
diff
changeset
|
861 return -1; |
3436
3c51658d6846
We didn't notice if messages were deleted directly from new/.
Timo Sirainen <tss@iki.fi>
parents:
3435
diff
changeset
|
862 if (ret == 0) |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
863 *lost_files_r = TRUE; |
3447
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
864 |
8554
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
865 i_assert(maildir_uidlist_is_locked(ctx->mbox->uidlist) || |
c438437b709a
maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota).
Timo Sirainen <tss@iki.fi>
parents:
7526
diff
changeset
|
866 lock_failure); |
2173
9438951e243f
Don't crash if we're syncing last commit to maildir, but some file was lost
Timo Sirainen <tss@iki.fi>
parents:
2140
diff
changeset
|
867 } |
1984 | 868 |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
869 if (find_uid != NULL && *find_uid != 0) { |
9017
0aa17f3e4a6d
Maildir: Handle uidlist errors better.
Timo Sirainen <tss@iki.fi>
parents:
8997
diff
changeset
|
870 ret = maildir_uidlist_lookup_nosync(ctx->mbox->uidlist, |
0aa17f3e4a6d
Maildir: Handle uidlist errors better.
Timo Sirainen <tss@iki.fi>
parents:
8997
diff
changeset
|
871 *find_uid, &flags, &fname); |
0aa17f3e4a6d
Maildir: Handle uidlist errors better.
Timo Sirainen <tss@iki.fi>
parents:
8997
diff
changeset
|
872 if (ret < 0) |
0aa17f3e4a6d
Maildir: Handle uidlist errors better.
Timo Sirainen <tss@iki.fi>
parents:
8997
diff
changeset
|
873 return -1; |
0aa17f3e4a6d
Maildir: Handle uidlist errors better.
Timo Sirainen <tss@iki.fi>
parents:
8997
diff
changeset
|
874 if (ret == 0) { |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
875 /* UID is expunged */ |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
876 *find_uid = 0; |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
877 } else if ((flags & MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) == 0) { |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
878 /* we didn't find it, possibly expunged? */ |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
879 *find_uid = 0; |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
880 } |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
881 } |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
882 |
9018
0bb192fe0abd
Maildir: More fixes to uidlist handling.
Timo Sirainen <tss@iki.fi>
parents:
9017
diff
changeset
|
883 return maildir_uidlist_sync_deinit(&ctx->uidlist_sync_ctx, TRUE); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
884 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
885 |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
886 int maildir_storage_sync_force(struct maildir_mailbox *mbox, uint32_t uid) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
887 { |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
888 struct maildir_sync_context *ctx; |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
889 bool lost_files; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
890 int ret; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
891 |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
892 T_BEGIN { |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
893 ctx = maildir_sync_context_new(mbox, MAILBOX_SYNC_FLAG_FAST); |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
894 ret = maildir_sync_context(ctx, TRUE, &uid, &lost_files); |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
895 maildir_sync_deinit(ctx); |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
896 } T_END; |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
897 |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
898 if (uid != 0) { |
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
899 /* maybe it's expunged. check again. */ |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
900 T_BEGIN { |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
901 ctx = maildir_sync_context_new(mbox, 0); |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
902 ret = maildir_sync_context(ctx, TRUE, NULL, |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
903 &lost_files); |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
904 maildir_sync_deinit(ctx); |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
905 } T_END; |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
906 } |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
907 return ret; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
908 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
909 |
2322
aae574ed7f4c
Broke mailbox_sync() into iterator.
Timo Sirainen <tss@iki.fi>
parents:
2320
diff
changeset
|
910 struct mailbox_sync_context * |
aae574ed7f4c
Broke mailbox_sync() into iterator.
Timo Sirainen <tss@iki.fi>
parents:
2320
diff
changeset
|
911 maildir_storage_sync_init(struct mailbox *box, enum mailbox_sync_flags flags) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
912 { |
3279
b698ae839a18
Moved mbox/maildir-specific variables from struct index_mailbox to
Timo Sirainen <tss@iki.fi>
parents:
3203
diff
changeset
|
913 struct maildir_mailbox *mbox = (struct maildir_mailbox *)box; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
914 struct maildir_sync_context *ctx; |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
915 bool lost_files; |
2322
aae574ed7f4c
Broke mailbox_sync() into iterator.
Timo Sirainen <tss@iki.fi>
parents:
2320
diff
changeset
|
916 int ret = 0; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
917 |
4894
24afafbfe47b
Make sure the mailbox is opened when transaction is started (fixes deliver).
Timo Sirainen <tss@iki.fi>
parents:
4848
diff
changeset
|
918 if (!box->opened) |
24afafbfe47b
Make sure the mailbox is opened when transaction is started (fixes deliver).
Timo Sirainen <tss@iki.fi>
parents:
4848
diff
changeset
|
919 index_storage_mailbox_open(&mbox->ibox); |
4848
967de900c73a
Mailbox list indexing and related changes. Currently works only with
Timo Sirainen <tss@iki.fi>
parents:
4774
diff
changeset
|
920 |
7291
db65d921b0e1
Check and update sync_last_check in one common function.
Timo Sirainen <tss@iki.fi>
parents:
7226
diff
changeset
|
921 if (index_mailbox_want_full_sync(&mbox->ibox, flags)) { |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
922 T_BEGIN { |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
923 ctx = maildir_sync_context_new(mbox, flags); |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
924 ret = maildir_sync_context(ctx, FALSE, NULL, |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
925 &lost_files); |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
926 maildir_sync_deinit(ctx); |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
927 } T_END; |
3436
3c51658d6846
We didn't notice if messages were deleted directly from new/.
Timo Sirainen <tss@iki.fi>
parents:
3435
diff
changeset
|
928 |
4152
e2edd333c473
Added MAILBOX_OPEN_KEEP_LOCKED flag to mailbox opening and implemented it
Timo Sirainen <tss@iki.fi>
parents:
4126
diff
changeset
|
929 i_assert(!maildir_uidlist_is_locked(mbox->uidlist) || |
e2edd333c473
Added MAILBOX_OPEN_KEEP_LOCKED flag to mailbox opening and implemented it
Timo Sirainen <tss@iki.fi>
parents:
4126
diff
changeset
|
930 mbox->ibox.keep_locked); |
3447
96de02ea7482
Keywords are stored in maildir filename and maildir-keywords file
Timo Sirainen <tss@iki.fi>
parents:
3446
diff
changeset
|
931 |
6452
a62923d3c969
Don't sync mailbox immediately when committing transactions. When it's done
Timo Sirainen <tss@iki.fi>
parents:
6429
diff
changeset
|
932 if (lost_files) { |
3436
3c51658d6846
We didn't notice if messages were deleted directly from new/.
Timo Sirainen <tss@iki.fi>
parents:
3435
diff
changeset
|
933 /* lost some files from new/, see if thery're in cur/ */ |
6886
82420ed23379
If doing a lockless forced sync, track the UID we're trying to find. If we
Timo Sirainen <tss@iki.fi>
parents:
6884
diff
changeset
|
934 ret = maildir_storage_sync_force(mbox, 0); |
3436
3c51658d6846
We didn't notice if messages were deleted directly from new/.
Timo Sirainen <tss@iki.fi>
parents:
3435
diff
changeset
|
935 } |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
936 } |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
937 |
8876
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
938 if (mbox->very_dirty_syncs) { |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
939 struct mail_index_view_sync_ctx *sync_ctx; |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
940 bool b; |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
941 |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
942 if (mbox->flags_view == NULL) { |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
943 mbox->flags_view = |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
944 mail_index_view_open(mbox->ibox.index); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
945 } |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
946 sync_ctx = mail_index_view_sync_begin(mbox->flags_view, |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
947 MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
948 if (mail_index_view_sync_commit(&sync_ctx, &b) < 0) { |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
949 mail_storage_set_index_error(&mbox->ibox); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
950 ret = -1; |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
951 } |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
952 /* make sure the map stays in private memory */ |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
953 if (mbox->flags_view->map->refcount > 1) { |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
954 struct mail_index_map *map; |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
955 |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
956 map = mail_index_map_clone(mbox->flags_view->map); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
957 mail_index_unmap(&mbox->flags_view->map); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
958 mbox->flags_view->map = map; |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
959 } |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
960 mail_index_record_map_move_to_private(mbox->flags_view->map); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
961 mail_index_map_move_to_memory(mbox->flags_view->map); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
962 maildir_uidlist_set_all_nonsynced(mbox->uidlist); |
88aeadb32151
Added maildir_very_dirty_syncs setting.
Timo Sirainen <tss@iki.fi>
parents:
8847
diff
changeset
|
963 } |
2322
aae574ed7f4c
Broke mailbox_sync() into iterator.
Timo Sirainen <tss@iki.fi>
parents:
2320
diff
changeset
|
964 return index_mailbox_sync_init(box, flags, ret < 0); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
965 } |
3472
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
966 |
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
967 int maildir_sync_is_synced(struct maildir_mailbox *mbox) |
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
968 { |
3863
55df57c028d4
Added "bool" type and changed all ints that were used as booleans to bool.
Timo Sirainen <tss@iki.fi>
parents:
3821
diff
changeset
|
969 bool new_changed, cur_changed; |
55df57c028d4
Added "bool" type and changed all ints that were used as booleans to bool.
Timo Sirainen <tss@iki.fi>
parents:
3821
diff
changeset
|
970 int ret; |
3472
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
971 |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
972 T_BEGIN { |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
973 const char *new_dir, *cur_dir; |
3472
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
974 |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
975 new_dir = t_strconcat(mbox->path, "/new", NULL); |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
976 cur_dir = t_strconcat(mbox->path, "/cur", NULL); |
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
977 |
7066
048325df1ec4
If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands),
Timo Sirainen <tss@iki.fi>
parents:
7046
diff
changeset
|
978 ret = maildir_sync_quick_check(mbox, FALSE, new_dir, cur_dir, |
6940
414c9d631a81
Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents:
6933
diff
changeset
|
979 &new_changed, &cur_changed); |
7226
e6693a0ec8e1
Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents:
7086
diff
changeset
|
980 } T_END; |
3472
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
981 return ret < 0 ? -1 : (!new_changed && !cur_changed); |
db29cc6754d5
Store new/ directory's timestamp in sync_size header in index (kludgy..).
Timo Sirainen <tss@iki.fi>
parents:
3470
diff
changeset
|
982 } |