dovecot/core-2.2: src/lib-index/mail-index-view-sync.c annotate

annotate src/lib-index/mail-index-view-sync.c @ 22576:707ae9de3812

lib: istream-multiplex - Minor code cleanup Avoid propagating the error twice, and avoid any confusion about what "got" actually contains.

author	Timo Sirainen <timo.sirainen@dovecot.fi>
date	Thu, 05 Oct 2017 20:24:11 +0300
parents	e8d999dd8043
children	cb108f786fb4

rev	line source
21390 2e2563132d5f Updated copyright notices to include the year 2017. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21389 diff changeset	1 /* Copyright (c) 2003-2017 Dovecot authors, see the included COPYING file */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
3470 346a494c2feb Moved array declaration to array-decl.h and include it in lib.h. So array.h Timo Sirainen <tss@iki.fi> parents: 3358 diff changeset	4 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "buffer.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	6 #include "mail-index-view-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	7 #include "mail-index-sync-private.h"
7620 4b8c1c164d8f Initial CONDSTORE support. Timo Sirainen <tss@iki.fi> parents: 7330 diff changeset	8 #include "mail-index-modseq.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	9 #include "mail-transaction-log.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	10
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	11
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	12 struct mail_index_view_sync_ctx {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	13 struct mail_index_view *view;
5768 35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	14 enum mail_index_view_sync_flags flags;
2803 bbfb6a094bb1 Crashfix Timo Sirainen <tss@iki.fi> parents: 2762 diff changeset	15 struct mail_index_sync_map_ctx sync_map_ctx;
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	16
7623 1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	17 /* After syncing view, map is replaced with sync_new_map. */
1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	18 struct mail_index_map *sync_new_map;
1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	19
4688 764bd841e0ae Try to avoid sending duplicate/useless flag updates. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	20 ARRAY_TYPE(seq_range) expunges;
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	21 unsigned int finish_min_msg_count;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	22
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	23 const struct mail_transaction_header *hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	24 const void *data;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	25
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	26 /* temporary variables while handling lost transaction logs: */
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	27 ARRAY_TYPE(keyword_indexes) lost_old_kw, lost_new_kw;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	28 buffer_t *lost_kw_buf;
7932 91758686277a View syncing: Keep track of highest modseq in views. If we lose transaction Timo Sirainen <tss@iki.fi> parents: 7885 diff changeset	29 uint32_t lost_new_ext_idx;
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	30 /* result of lost transaction logs: */
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	31 ARRAY_TYPE(seq_range) lost_flags;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	32 unsigned int lost_flag_idx;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	33
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	34 size_t data_offset;
6279 f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	35 unsigned int failed:1;
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	36 unsigned int sync_map_update:1;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	37 unsigned int skipped_expunges:1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	38 unsigned int last_read:1;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	39 unsigned int log_was_lost:1;
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	40 unsigned int hidden:1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	41 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	42
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5653 diff changeset	43 static int
6663 7d7f31de0826 Don't assert-crash if index was just reset. Timo Sirainen <tss@iki.fi> parents: 6458 diff changeset	44 view_sync_set_log_view_range(struct mail_index_view *view, bool sync_expunges,
20818 139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	45 bool reset_r, bool partial_sync_r)
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3825 diff changeset	46 {
5772 002aa9bbfcb2 Removed index->hdr. Use index->map->hdr instead. Timo Sirainen <tss@iki.fi> parents: 5771 diff changeset	47 const struct mail_index_header *hdr = &view->index->map->hdr;
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	48 uint32_t start_seq, end_seq;
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	49 uoff_t start_offset, end_offset;
18398 8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	50 const char *reason;
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3825 diff changeset	51 int ret;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3825 diff changeset	52
20818 139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	53 *partial_sync_r = FALSE;
139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	54
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	55 start_seq = view->log_file_expunge_seq;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	56 start_offset = view->log_file_expunge_offset;
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	57 end_seq = hdr->log_file_seq;
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	58 end_offset = hdr->log_file_head_offset;
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	59
10071 359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	60 if (end_seq < view->log_file_head_seq \|\|
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	61 (end_seq == view->log_file_head_seq &&
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	62 end_offset < view->log_file_head_offset)) {
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	63 mail_index_set_error(view->index,
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	64 "%s log position went backwards "
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	65 "(%u,%"PRIuUOFF_T" < %u,%"PRIuUOFF_T")",
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	66 view->index->filepath, end_seq, end_offset,
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	67 view->log_file_head_seq, view->log_file_head_offset);
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	68 return -1;
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	69 }
359a4e4f0892 lib-index: Give nicer error if index's log position suddenly goes backwards. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	70
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	71 for (;;) {
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	72 /* the view begins from the first non-synced transaction */
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	73 ret = mail_transaction_log_view_set(view->log_view,
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	74 start_seq, start_offset,
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	75 end_seq, end_offset,
18398 8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	76 reset_r, &reason);
22061 e8d999dd8043 lib-index: Add missing error logging for mail_index_view_sync_begin() Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	77 if (ret <= 0) {
e8d999dd8043 lib-index: Add missing error logging for mail_index_view_sync_begin() Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	78 mail_index_set_error(view->index,
e8d999dd8043 lib-index: Add missing error logging for mail_index_view_sync_begin() Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	79 "Failed to map view for %s: %s",
e8d999dd8043 lib-index: Add missing error logging for mail_index_view_sync_begin() Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	80 view->index->filepath, reason);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	81 return ret;
22061 e8d999dd8043 lib-index: Add missing error logging for mail_index_view_sync_begin() Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	82 }
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	83
6663 7d7f31de0826 Don't assert-crash if index was just reset. Timo Sirainen <tss@iki.fi> parents: 6458 diff changeset	84 if (!*reset_r \|\| sync_expunges)
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	85 break;
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	86
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	87 /* log was reset, but we don't want to sync expunges.
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	88 we can't do this, so sync only up to the reset. */
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	89 mail_transaction_log_view_get_prev_pos(view->log_view,
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	90 &end_seq, &end_offset);
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	91 end_seq--; end_offset = (uoff_t)-1;
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	92 if (end_seq < start_seq) {
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	93 /* we have only this reset log */
6458 cca479cb3ef0 Keep transaction logs referenced while we know we need them. This fixes Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	94 mail_transaction_log_view_clear(view->log_view,
cca479cb3ef0 Keep transaction logs referenced while we know we need them. This fixes Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	95 view->log_file_expunge_seq);
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	96 break;
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	97 }
20818 139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	98 *partial_sync_r = TRUE;
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3825 diff changeset	99 }
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	100 return 1;
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3825 diff changeset	101 }
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3825 diff changeset	102
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	103 static unsigned int
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	104 view_sync_expunges2seqs(struct mail_index_view_sync_ctx *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	105 {
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	106 struct mail_index_view *view = ctx->view;
4688 764bd841e0ae Try to avoid sending duplicate/useless flag updates. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	107 struct seq_range src, src_end, *dest;
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	108 unsigned int count, expunge_count = 0;
6001 2d81847eb3e6 Added new assert Timo Sirainen <tss@iki.fi> parents: 6000 diff changeset	109 uint32_t prev_seq = 0;
2033 4f6b1118a53d Transaction log contains only UIDs now, no more sequences which just mess up Timo Sirainen <tss@iki.fi> parents: 1988 diff changeset	110
4688 764bd841e0ae Try to avoid sending duplicate/useless flag updates. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	111 /* convert UIDs to sequences */
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	112 src = dest = array_get_modifiable(&ctx->expunges, &count);
3346 8e093a2cc8db Expunge code cleanups. Timo Sirainen <tss@iki.fi> parents: 3278 diff changeset	113 src_end = src + count;
3187 aa83063864c7 Code cleanups Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	114 for (; src != src_end; src++) {
6677 90bcf480312d Renamed mail_index_lookup_uid_range() to mail_index_lookup_seq_range(), made Timo Sirainen <tss@iki.fi> parents: 6675 diff changeset	115 if (!mail_index_lookup_seq_range(view, src->seq1, src->seq2,
90bcf480312d Renamed mail_index_lookup_uid_range() to mail_index_lookup_seq_range(), made Timo Sirainen <tss@iki.fi> parents: 6675 diff changeset	116 &dest->seq1, &dest->seq2))
3346 8e093a2cc8db Expunge code cleanups. Timo Sirainen <tss@iki.fi> parents: 3278 diff changeset	117 count--;
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	118 else {
6001 2d81847eb3e6 Added new assert Timo Sirainen <tss@iki.fi> parents: 6000 diff changeset	119 i_assert(dest->seq1 > prev_seq);
2d81847eb3e6 Added new assert Timo Sirainen <tss@iki.fi> parents: 6000 diff changeset	120 prev_seq = dest->seq2;
2d81847eb3e6 Added new assert Timo Sirainen <tss@iki.fi> parents: 6000 diff changeset	121
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	122 expunge_count += dest->seq2 - dest->seq1 + 1;
3187 aa83063864c7 Code cleanups Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	123 dest++;
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	124 }
3187 aa83063864c7 Code cleanups Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	125 }
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	126 array_delete(&ctx->expunges, count,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	127 array_count(&ctx->expunges) - count);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	128 return expunge_count;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	129 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	130
9621 8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	131 static void
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	132 view_sync_add_expunge_range(ARRAY_TYPE(seq_range) *dest,
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	133 const struct seq_range *src, size_t src_size)
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	134 {
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	135 unsigned int i, src_count;
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	136
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	137 i_assert(src_size % sizeof(*src) == 0);
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	138
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	139 src_count = src_size / sizeof(*src);
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	140 for (i = 0; i < src_count; i++)
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	141 seq_range_array_add_range(dest, src[i].seq1, src[i].seq2);
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	142 }
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	143
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	144 static void
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	145 view_sync_add_expunge_guids(ARRAY_TYPE(seq_range) *dest,
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	146 const struct mail_transaction_expunge_guid *src,
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	147 size_t src_size)
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	148 {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	149 unsigned int i, src_count;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	150
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	151 i_assert(src_size % sizeof(*src) == 0);
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	152
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	153 src_count = src_size / sizeof(*src);
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	154 for (i = 0; i < src_count; i++)
14676 69ba6977bed8 seq_range_array_add() API changed. Added other functions to provide the less common use cases. Timo Sirainen <tss@iki.fi> parents: 14133 diff changeset	155 seq_range_array_add(dest, src[i].uid);
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	156 }
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	157
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	158 static int
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	159 view_sync_get_expunges(struct mail_index_view_sync_ctx *ctx,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	160 unsigned int *expunge_count_r)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	161 {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	162 struct mail_index_view *view = ctx->view;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	163 const struct mail_transaction_header *hdr;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	164 const void *data;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	165 int ret;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	166
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	167 /* get a list of expunge transactions. there may be some that we have
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	168 already synced, but it doesn't matter because they'll get dropped
9621 8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	169 out when converting to sequences. the uid ranges' validity has
8c17eb6c28d6 Mail index view syncs: Simplify code for getting list of expunges. Timo Sirainen <tss@iki.fi> parents: 8806 diff changeset	170 already been verified, so we can use them directly. */
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	171 mail_transaction_log_view_mark(view->log_view);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	172 while ((ret = mail_transaction_log_view_next(view->log_view,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	173 &hdr, &data)) > 0) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	174 if ((hdr->type & MAIL_TRANSACTION_EXTERNAL) == 0) {
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	175 /* skip expunge requests */
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	176 continue;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	177 }
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	178 if ((hdr->type & MAIL_TRANSACTION_EXPUNGE_GUID) != 0) {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	179 view_sync_add_expunge_guids(&ctx->expunges,
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	180 data, hdr->size);
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	181 } else if ((hdr->type & MAIL_TRANSACTION_EXPUNGE) != 0) {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	182 view_sync_add_expunge_range(&ctx->expunges,
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	183 data, hdr->size);
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	184 }
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	185 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	186 mail_transaction_log_view_rewind(view->log_view);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	187
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	188 *expunge_count_r = view_sync_expunges2seqs(ctx);
11038 f72884695084 lib-index: View syncing error handling fix. Timo Sirainen <tss@iki.fi> parents: 10582 diff changeset	189 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	190 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	191
6275 913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	192 static bool have_existing_expunges(struct mail_index_view *view,
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	193 const struct seq_range *range, size_t size)
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	194 {
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	195 const struct seq_range *range_end;
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	196 uint32_t seq1, seq2;
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	197
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	198 range_end = CONST_PTR_OFFSET(range, size);
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	199 for (; range != range_end; range++) {
6677 90bcf480312d Renamed mail_index_lookup_uid_range() to mail_index_lookup_seq_range(), made Timo Sirainen <tss@iki.fi> parents: 6675 diff changeset	200 if (mail_index_lookup_seq_range(view, range->seq1, range->seq2,
90bcf480312d Renamed mail_index_lookup_uid_range() to mail_index_lookup_seq_range(), made Timo Sirainen <tss@iki.fi> parents: 6675 diff changeset	201 &seq1, &seq2))
6275 913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	202 return TRUE;
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	203 }
6275 913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	204 return FALSE;
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	205 }
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	206
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	207 static bool
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	208 have_existing_guid_expunge(struct mail_index_view *view,
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	209 const struct mail_transaction_expunge_guid *expunges,
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	210 size_t size)
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	211 {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	212 const struct mail_transaction_expunge_guid *expunges_end;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	213 uint32_t seq;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	214
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	215 expunges_end = CONST_PTR_OFFSET(expunges, size);
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	216 for (; expunges != expunges_end; expunges++) {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	217 if (mail_index_lookup_seq(view, expunges->uid, &seq))
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	218 return TRUE;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	219 }
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	220 return FALSE;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	221 }
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	222
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	223 static bool view_sync_have_expunges(struct mail_index_view *view)
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	224 {
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	225 const struct mail_transaction_header *hdr;
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	226 const void *data;
6275 913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	227 bool have_expunges = FALSE;
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	228 int ret;
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	229
7812 d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset. Timo Sirainen <tss@iki.fi> parents: 7626 diff changeset	230 mail_transaction_log_view_mark(view->log_view);
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	231
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	232 while ((ret = mail_transaction_log_view_next(view->log_view,
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	233 &hdr, &data)) > 0) {
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	234 if ((hdr->type & MAIL_TRANSACTION_EXTERNAL) == 0) {
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	235 /* skip expunge requests */
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	236 continue;
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	237 }
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	238 if ((hdr->type & MAIL_TRANSACTION_EXPUNGE_GUID) != 0) {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	239 /* we have an expunge. see if it still exists. */
9971 309ef3136fa0 lib-index: Fixed v2.0 forwards compatibility support. Expunges were handled wrong. Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	240 if (have_existing_guid_expunge(view, data, hdr->size)) {
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	241 have_expunges = TRUE;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	242 break;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	243 }
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	244 } else if ((hdr->type & MAIL_TRANSACTION_EXPUNGE) != 0) {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	245 /* we have an expunge. see if it still exists. */
9971 309ef3136fa0 lib-index: Fixed v2.0 forwards compatibility support. Expunges were handled wrong. Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	246 if (have_existing_expunges(view, data, hdr->size)) {
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	247 have_expunges = TRUE;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	248 break;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	249 }
6275 913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	250 }
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	251 }
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	252
7812 d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset. Timo Sirainen <tss@iki.fi> parents: 7626 diff changeset	253 mail_transaction_log_view_rewind(view->log_view);
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	254
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	255 /* handle failures as having expunges (which is safer).
7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	256 we'll probably fail later. */
6275 913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	257 return ret < 0 \|\| have_expunges;
5863 7d1844cbfd98 If there are no expunges, sync to head map instead of keeping track of a Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	258 }
5268 debb7be008dc We skipped some external transactions that shouldn't have been skipped. Timo Sirainen <tss@iki.fi> parents: 5065 diff changeset	259
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	260 static int uint_cmp(const void p1, const void p2)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	261 {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	262 const unsigned int u1 = p1, u2 = p2;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	263
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	264 if (u1 < u2)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	265 return -1;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	266 if (u1 > u2)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	267 return 1;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	268 return 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	269 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	270
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	271 static bool view_sync_lost_keywords_equal(struct mail_index_view_sync_ctx *ctx)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	272 {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	273 unsigned int old_idx, new_idx;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	274 unsigned int old_count, new_count;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	275
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	276 old_idx = array_get_modifiable(&ctx->lost_old_kw, &old_count);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	277 new_idx = array_get_modifiable(&ctx->lost_new_kw, &new_count);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	278 if (old_count != new_count)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	279 return FALSE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	280
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	281 qsort(old_idx, old_count, sizeof(*old_idx), uint_cmp);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	282 qsort(new_idx, new_count, sizeof(*new_idx), uint_cmp);
16713 00d0856b4a0b lib-index: Minor fix to regenerating missing transaction log parts. Timo Sirainen <tss@iki.fi> parents: 15715 diff changeset	283 return memcmp(old_idx, new_idx, old_count * sizeof(*old_idx)) == 0;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	284 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	285
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	286 static int view_sync_update_keywords(struct mail_index_view_sync_ctx *ctx,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	287 uint32_t uid)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	288 {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	289 struct mail_transaction_header thdr;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	290 struct mail_transaction_keyword_update kw_up;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	291 const unsigned int *kw_idx;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	292 const char const kw_names;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	293 unsigned int i, count;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	294
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	295 kw_idx = array_get(&ctx->lost_new_kw, &count);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	296 if (count == 0)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	297 return 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	298 kw_names = array_idx(&ctx->view->index->keywords, 0);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	299
21389 59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(p)) with the new i_zero() macro. Stephan Bosch <stephan.bosch@dovecot.fi>* parents: 20820 diff changeset	300 i_zero(&thdr);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	301 thdr.type = MAIL_TRANSACTION_KEYWORD_UPDATE \| MAIL_TRANSACTION_EXTERNAL;
21389 59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(p)) with the new i_zero() macro. Stephan Bosch <stephan.bosch@dovecot.fi>* parents: 20820 diff changeset	302 i_zero(&kw_up);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	303 kw_up.modify_type = MODIFY_ADD;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	304 /* add new flags one by one */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	305 for (i = 0; i < count; i++) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	306 kw_up.name_size = strlen(kw_names[kw_idx[i]]);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	307 buffer_set_used_size(ctx->lost_kw_buf, 0);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	308 buffer_append(ctx->lost_kw_buf, &kw_up, sizeof(kw_up));
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	309 buffer_append(ctx->lost_kw_buf, kw_names[kw_idx[i]],
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	310 kw_up.name_size);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	311 if (ctx->lost_kw_buf->used % 4 != 0) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	312 buffer_append_zero(ctx->lost_kw_buf,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	313 4 - ctx->lost_kw_buf->used % 4);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	314 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	315 buffer_append(ctx->lost_kw_buf, &uid, sizeof(uid));
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	316 buffer_append(ctx->lost_kw_buf, &uid, sizeof(uid));
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	317
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	318 thdr.size = ctx->lost_kw_buf->used;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	319 if (mail_index_sync_record(&ctx->sync_map_ctx, &thdr,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	320 ctx->lost_kw_buf->data) < 0)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	321 return -1;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	322 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	323 return 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	324 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	325
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	326 static int view_sync_apply_lost_changes(struct mail_index_view_sync_ctx *ctx,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	327 uint32_t old_seq, uint32_t new_seq)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	328 {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	329 struct mail_index_map *old_map = ctx->view->map;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	330 struct mail_index_map *new_map = ctx->view->index->map;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	331 const struct mail_index_record old_rec, new_rec;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	332 struct mail_transaction_header thdr;
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	333 const struct mail_index_ext *ext;
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	334 const uint64_t *modseqp;
7932 91758686277a View syncing: Keep track of highest modseq in views. If we lose transaction Timo Sirainen <tss@iki.fi> parents: 7885 diff changeset	335 uint64_t new_modseq;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	336 bool changed = FALSE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	337
17428 4b1117b5cd8e mail-index: trivial robomatic migration from _MAP_IDX to new helper Phil Carmody <phil@dovecot.fi> parents: 17130 diff changeset	338 old_rec = MAIL_INDEX_REC_AT_SEQ(old_map, old_seq);
4b1117b5cd8e mail-index: trivial robomatic migration from _MAP_IDX to new helper Phil Carmody <phil@dovecot.fi> parents: 17130 diff changeset	339 new_rec = MAIL_INDEX_REC_AT_SEQ(new_map, new_seq);
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	340
21389 59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(p)) with the new i_zero() macro. Stephan Bosch <stephan.bosch@dovecot.fi>* parents: 20820 diff changeset	341 i_zero(&thdr);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	342 if (old_rec->flags != new_rec->flags) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	343 struct mail_transaction_flag_update flag_update;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	344
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	345 /* check this before syncing the record, since it updates
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	346 old_rec. */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	347 if ((old_rec->flags & MAIL_INDEX_FLAGS_MASK) !=
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	348 (new_rec->flags & MAIL_INDEX_FLAGS_MASK))
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	349 changed = TRUE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	350
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	351 thdr.type = MAIL_TRANSACTION_FLAG_UPDATE \|
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	352 MAIL_TRANSACTION_EXTERNAL;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	353 thdr.size = sizeof(flag_update);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	354
21389 59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(p)) with the new i_zero() macro. Stephan Bosch <stephan.bosch@dovecot.fi>* parents: 20820 diff changeset	355 i_zero(&flag_update);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	356 flag_update.uid1 = flag_update.uid2 = new_rec->uid;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	357 flag_update.add_flags = new_rec->flags;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	358 flag_update.remove_flags = ~new_rec->flags & 0xff;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	359 if (mail_index_sync_record(&ctx->sync_map_ctx, &thdr,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	360 &flag_update) < 0)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	361 return -1;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	362 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	363
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	364 mail_index_map_lookup_keywords(old_map, old_seq, &ctx->lost_old_kw);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	365 mail_index_map_lookup_keywords(new_map, new_seq, &ctx->lost_new_kw);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	366 if (!view_sync_lost_keywords_equal(ctx)) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	367 struct mail_transaction_keyword_reset kw_reset;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	368
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	369 thdr.type = MAIL_TRANSACTION_KEYWORD_RESET \|
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	370 MAIL_TRANSACTION_EXTERNAL;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	371 thdr.size = sizeof(kw_reset);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	372
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	373 /* remove all old flags by resetting them */
21389 59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(p)) with the new i_zero() macro. Stephan Bosch <stephan.bosch@dovecot.fi>* parents: 20820 diff changeset	374 i_zero(&kw_reset);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	375 kw_reset.uid1 = kw_reset.uid2 = new_rec->uid;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	376 if (mail_index_sync_record(&ctx->sync_map_ctx, &thdr,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	377 &kw_reset) < 0)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	378 return -1;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	379
14682 d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this. Timo Sirainen <tss@iki.fi> parents: 14676 diff changeset	380 if (view_sync_update_keywords(ctx, new_rec->uid) < 0)
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this. Timo Sirainen <tss@iki.fi> parents: 14676 diff changeset	381 return -1;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	382 changed = TRUE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	383 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	384
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	385 if (changed) {
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	386 /* flags or keywords changed */
7932 91758686277a View syncing: Keep track of highest modseq in views. If we lose transaction Timo Sirainen <tss@iki.fi> parents: 7885 diff changeset	387 } else if (ctx->view->highest_modseq != 0 &&
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	388 ctx->lost_new_ext_idx != (uint32_t)-1) {
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	389 /* if modseq has changed include this message in changed flags
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	390 list, even if we didn't see any changes above. */
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	391 ext = array_idx(&new_map->extensions, ctx->lost_new_ext_idx);
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	392 modseqp = CONST_PTR_OFFSET(new_rec, ext->record_offset);
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	393 new_modseq = *modseqp;
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	394
7932 91758686277a View syncing: Keep track of highest modseq in views. If we lose transaction Timo Sirainen <tss@iki.fi> parents: 7885 diff changeset	395 if (new_modseq > ctx->view->highest_modseq)
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	396 changed = TRUE;
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	397 }
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	398
7932 91758686277a View syncing: Keep track of highest modseq in views. If we lose transaction Timo Sirainen <tss@iki.fi> parents: 7885 diff changeset	399 /* without modseqs lost_flags isn't updated perfectly correctly, because
91758686277a View syncing: Keep track of highest modseq in views. If we lose transaction Timo Sirainen <tss@iki.fi> parents: 7885 diff changeset	400 by the time we're comparing old flags it may have changed from what
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	401 we last sent to the client (because the map is shared). This could
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	402 be avoided by always keeping a private copy of the map in the view,
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	403 but that's a waste of memory for as rare of a problem as this. */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	404 if (changed)
14676 69ba6977bed8 seq_range_array_add() API changed. Added other functions to provide the less common use cases. Timo Sirainen <tss@iki.fi> parents: 14133 diff changeset	405 seq_range_array_add(&ctx->lost_flags, new_rec->uid);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	406 return 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	407 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	408
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	409 static int
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	410 view_sync_get_log_lost_changes(struct mail_index_view_sync_ctx *ctx,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	411 unsigned int *expunge_count_r)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	412 {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	413 struct mail_index_view *view = ctx->view;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	414 struct mail_index_map *old_map = view->map;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	415 struct mail_index_map *new_map = view->index->map;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	416 const unsigned int old_count = old_map->hdr.messages_count;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	417 const unsigned int new_count = new_map->hdr.messages_count;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	418 const struct mail_index_record old_rec, new_rec;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	419 struct mail_transaction_header thdr;
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	420 uint32_t seqi, seqj;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	421
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	422 /* we don't update the map in the same order as it's typically done.
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	423 map->rec_map may already have some messages appended that we don't
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	424 want. get an atomic map to make sure these get removed. */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	425 (void)mail_index_sync_get_atomic_map(&ctx->sync_map_ctx);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	426
7885 ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	427 if (!mail_index_map_get_ext_idx(new_map, view->index->modseq_ext_id,
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	428 &ctx->lost_new_ext_idx))
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	429 ctx->lost_new_ext_idx = (uint32_t)-1;
ad087a4c480f Mailbox view sync: If modseqs are enabled, use them as well as direct Timo Sirainen <tss@iki.fi> parents: 7883 diff changeset	430
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	431 i_array_init(&ctx->lost_flags, 64);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	432 t_array_init(&ctx->lost_old_kw, 32);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	433 t_array_init(&ctx->lost_new_kw, 32);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	434 ctx->lost_kw_buf = buffer_create_dynamic(pool_datastack_create(), 128);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	435
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	436 /* handle expunges and sync flags */
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	437 seqi = seqj = 1;
20428 fe61bfdf25ac lib-index: Fixed view syncing when changes couldn't be read from transaction logs Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	438 while (seqi <= old_count && seqj <= new_count) {
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	439 old_rec = MAIL_INDEX_REC_AT_SEQ(old_map, seqi);
26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	440 new_rec = MAIL_INDEX_REC_AT_SEQ(new_map, seqj);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	441 if (old_rec->uid == new_rec->uid) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	442 /* message found - check if flags have changed */
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	443 if (view_sync_apply_lost_changes(ctx, seqi, seqj) < 0)
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	444 return -1;
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	445 seqi++; seqj++;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	446 } else if (old_rec->uid < new_rec->uid) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	447 /* message expunged */
14676 69ba6977bed8 seq_range_array_add() API changed. Added other functions to provide the less common use cases. Timo Sirainen <tss@iki.fi> parents: 14133 diff changeset	448 seq_range_array_add(&ctx->expunges, old_rec->uid);
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	449 seqi++;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	450 } else {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	451 /* new message appeared out of nowhere */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	452 mail_index_set_error(view->index,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	453 "%s view is inconsistent: "
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	454 "uid=%u inserted in the middle of mailbox",
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	455 view->index->filepath, new_rec->uid);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	456 return -1;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	457 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	458 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	459 /* if there are old messages left, they're all expunged */
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	460 for (; seqi <= old_count; seqi++) {
26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	461 old_rec = MAIL_INDEX_REC_AT_SEQ(old_map, seqi);
14676 69ba6977bed8 seq_range_array_add() API changed. Added other functions to provide the less common use cases. Timo Sirainen <tss@iki.fi> parents: 14133 diff changeset	462 seq_range_array_add(&ctx->expunges, old_rec->uid);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	463 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	464 /* if there are new messages left, they're all new messages */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	465 thdr.type = MAIL_TRANSACTION_APPEND \| MAIL_TRANSACTION_EXTERNAL;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	466 thdr.size = sizeof(*new_rec);
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	467 for (; seqj <= new_count; seqj++) {
26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	468 new_rec = MAIL_INDEX_REC_AT_SEQ(new_map, seqj);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	469 if (mail_index_sync_record(&ctx->sync_map_ctx,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	470 &thdr, new_rec) < 0)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	471 return -1;
17432 26e097feb136 mail-index: use _REC_AT_SEQ not _MAP_IDX in view_sync_get_log_lost_changes Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	472 mail_index_map_lookup_keywords(new_map, seqj,
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	473 &ctx->lost_new_kw);
14682 d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this. Timo Sirainen <tss@iki.fi> parents: 14676 diff changeset	474 if (view_sync_update_keywords(ctx, new_rec->uid) < 0)
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this. Timo Sirainen <tss@iki.fi> parents: 14676 diff changeset	475 return -1;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	476 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	477 *expunge_count_r = view_sync_expunges2seqs(ctx);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	478
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	479 /* we have no idea how far we've synced - make sure these aren't used */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	480 old_map->hdr.log_file_seq = 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	481 old_map->hdr.log_file_head_offset = 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	482 old_map->hdr.log_file_tail_offset = 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	483
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	484 if ((ctx->flags & MAIL_INDEX_VIEW_SYNC_FLAG_NOEXPUNGES) != 0) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	485 array_clear(&ctx->expunges);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	486 ctx->skipped_expunges = *expunge_count_r > 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	487 } else {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	488 view->log_file_head_seq = new_map->hdr.log_file_seq;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	489 view->log_file_head_offset = new_map->hdr.log_file_head_offset;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	490 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	491 return 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	492 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	493
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	494 static int mail_index_view_sync_init_fix(struct mail_index_view_sync_ctx *ctx)
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	495 {
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	496 struct mail_index_view *view = ctx->view;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	497 uint32_t seq;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	498 uoff_t offset;
18398 8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	499 const char *reason;
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	500 bool reset;
18398 8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	501 int ret;
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	502
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	503 /* replace the view's map */
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	504 view->index->map->refcount++;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	505 mail_index_unmap(&view->map);
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	506 view->map = view->index->map;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	507
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	508 /* update log positions */
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	509 view->log_file_head_seq = seq = view->map->hdr.log_file_seq;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	510 view->log_file_head_offset = offset =
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	511 view->map->hdr.log_file_head_offset;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	512
18398 8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	513 ret = mail_transaction_log_view_set(view->log_view, seq, offset,
8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	514 seq, offset, &reset, &reason);
22061 e8d999dd8043 lib-index: Add missing error logging for mail_index_view_sync_begin() Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	515 if (ret <= 0) {
18398 8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	516 mail_index_set_error(view->index, "Failed to fix view for %s: %s",
8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	517 view->index->filepath, reason);
22061 e8d999dd8043 lib-index: Add missing error logging for mail_index_view_sync_begin() Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	518 return ret;
18398 8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	519 }
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	520 view->inconsistent = FALSE;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	521 return 0;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	522 }
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	523
7872 3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	524 struct mail_index_view_sync_ctx *
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	525 mail_index_view_sync_begin(struct mail_index_view *view,
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	526 enum mail_index_view_sync_flags flags)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	527 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	528 struct mail_index_view_sync_ctx *ctx;
11039 0f98525e4567 Removed dead code. Timo Sirainen <tss@iki.fi> parents: 11038 diff changeset	529 struct mail_index_map *tmp_map;
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	530 unsigned int expunge_count = 0;
20818 139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	531 bool reset, partial_sync, sync_expunges, have_expunges;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	532 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	533
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	534 i_assert(!view->syncing);
3220 0220c7d37fc8 Added assert. Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	535 i_assert(view->transactions == 0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	536
7872 3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	537 view->syncing = TRUE;
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	538
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	539 /* Syncing the view invalidates all previous looked up records.
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	540 Unreference the mappings this view keeps because of them. */
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	541 mail_index_view_unref_maps(view);
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	542
7872 3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	543 ctx = i_new(struct mail_index_view_sync_ctx, 1);
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	544 ctx->view = view;
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	545 ctx->flags = flags;
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	546
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	547 sync_expunges = (flags & MAIL_INDEX_VIEW_SYNC_FLAG_NOEXPUNGES) == 0;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	548 if (sync_expunges)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	549 i_array_init(&ctx->expunges, 64);
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	550 if ((flags & MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT) != 0) {
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	551 /* just get this view synced - don't return anything */
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	552 i_assert(sync_expunges);
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	553 if (mail_index_view_sync_init_fix(ctx) < 0)
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	554 ctx->failed = TRUE;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	555 return ctx;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	556 }
6377 29f427039e00 fsck won't fail anymore with "corrupted index", all problems are fixed. Timo Sirainen <tss@iki.fi> parents: 6279 diff changeset	557 if (mail_index_view_is_inconsistent(view)) {
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	558 mail_index_set_error(view->index, "%s view is inconsistent",
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	559 view->index->filepath);
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	560 ctx->failed = TRUE;
af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	561 return ctx;
6377 29f427039e00 fsck won't fail anymore with "corrupted index", all problems are fixed. Timo Sirainen <tss@iki.fi> parents: 6279 diff changeset	562 }
29f427039e00 fsck won't fail anymore with "corrupted index", all problems are fixed. Timo Sirainen <tss@iki.fi> parents: 6279 diff changeset	563
20818 139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	564 ret = view_sync_set_log_view_range(view, sync_expunges, &reset, &partial_sync);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	565 if (ret < 0) {
7874 650330db2f99 mail_index_view_sync_begin() code cleanup. Timo Sirainen <tss@iki.fi> parents: 7873 diff changeset	566 ctx->failed = TRUE;
650330db2f99 mail_index_view_sync_begin() code cleanup. Timo Sirainen <tss@iki.fi> parents: 7873 diff changeset	567 return ctx;
650330db2f99 mail_index_view_sync_begin() code cleanup. Timo Sirainen <tss@iki.fi> parents: 7873 diff changeset	568 }
650330db2f99 mail_index_view_sync_begin() code cleanup. Timo Sirainen <tss@iki.fi> parents: 7873 diff changeset	569
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	570 if (ret == 0) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	571 ctx->log_was_lost = TRUE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	572 if (!sync_expunges)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	573 i_array_init(&ctx->expunges, 64);
8530 60ea101c7534 Crashfix to handling missing transaction logs. Timo Sirainen <tss@iki.fi> parents: 8131 diff changeset	574 mail_index_sync_map_init(&ctx->sync_map_ctx, view,
60ea101c7534 Crashfix to handling missing transaction logs. Timo Sirainen <tss@iki.fi> parents: 8131 diff changeset	575 MAIL_INDEX_SYNC_HANDLER_VIEW);
60ea101c7534 Crashfix to handling missing transaction logs. Timo Sirainen <tss@iki.fi> parents: 8131 diff changeset	576 ret = view_sync_get_log_lost_changes(ctx, &expunge_count);
60ea101c7534 Crashfix to handling missing transaction logs. Timo Sirainen <tss@iki.fi> parents: 8131 diff changeset	577 mail_index_modseq_sync_end(&ctx->sync_map_ctx.modseq_ctx);
60ea101c7534 Crashfix to handling missing transaction logs. Timo Sirainen <tss@iki.fi> parents: 8131 diff changeset	578 mail_index_sync_map_deinit(&ctx->sync_map_ctx);
60ea101c7534 Crashfix to handling missing transaction logs. Timo Sirainen <tss@iki.fi> parents: 8131 diff changeset	579 if (ret < 0) {
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	580 mail_index_set_error(view->index,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	581 "%s view syncing failed to apply changes",
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	582 view->index->filepath);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	583 view->inconsistent = TRUE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	584 ctx->failed = TRUE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	585 return ctx;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	586 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	587 have_expunges = expunge_count > 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	588 } else if (sync_expunges) {
6682 53173ab706a0 Fixes to MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT handling. Timo Sirainen <tss@iki.fi> parents: 6677 diff changeset	589 /* get list of all expunges first */
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	590 if (view_sync_get_expunges(ctx, &expunge_count) < 0) {
7872 3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	591 ctx->failed = TRUE;
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	592 return ctx;
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	593 }
7876 94fe6f7747da Mail index view sync: Don't look for expunges if we already did it once. Timo Sirainen <tss@iki.fi> parents: 7875 diff changeset	594 have_expunges = expunge_count > 0;
94fe6f7747da Mail index view sync: Don't look for expunges if we already did it once. Timo Sirainen <tss@iki.fi> parents: 7875 diff changeset	595 } else {
94fe6f7747da Mail index view sync: Don't look for expunges if we already did it once. Timo Sirainen <tss@iki.fi> parents: 7875 diff changeset	596 have_expunges = view_sync_have_expunges(view);
5722 9c53047d3b2c Changed mail_index_view_sync_begin() to take enum mail_index_view_sync_type. Timo Sirainen <tss@iki.fi> parents: 5689 diff changeset	597 }
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	598
7873 af9d5fd1071c mail_index_view_sync_begin() _FLAG_FIX_INCONSISTENT code cleanup. Timo Sirainen <tss@iki.fi> parents: 7872 diff changeset	599 ctx->finish_min_msg_count = reset ? 0 :
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	600 view->map->hdr.messages_count - expunge_count;
20820 0901db0db1c4 lib-index: If view syncing sees index is reset, always mark the view as inconsistent. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20819 diff changeset	601 if (reset) {
6673 c5e4a7b75586 Fixes to handling inconsistent views after index reset. Timo Sirainen <tss@iki.fi> parents: 6663 diff changeset	602 view->inconsistent = TRUE;
c5e4a7b75586 Fixes to handling inconsistent views after index reset. Timo Sirainen <tss@iki.fi> parents: 6663 diff changeset	603 mail_index_set_error(view->index,
c5e4a7b75586 Fixes to handling inconsistent views after index reset. Timo Sirainen <tss@iki.fi> parents: 6663 diff changeset	604 "%s reset, view is now inconsistent",
c5e4a7b75586 Fixes to handling inconsistent views after index reset. Timo Sirainen <tss@iki.fi> parents: 6663 diff changeset	605 view->index->filepath);
20819 e25ae081b1eb lib-index: When finding a view is reset, don't attempt any further syncing. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20818 diff changeset	606 ctx->failed = TRUE;
e25ae081b1eb lib-index: When finding a view is reset, don't attempt any further syncing. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20818 diff changeset	607 return ctx;
6673 c5e4a7b75586 Fixes to handling inconsistent views after index reset. Timo Sirainen <tss@iki.fi> parents: 6663 diff changeset	608 }
c5e4a7b75586 Fixes to handling inconsistent views after index reset. Timo Sirainen <tss@iki.fi> parents: 6663 diff changeset	609
20818 139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	610 if (!have_expunges && !partial_sync) {
7623 1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	611 /* no expunges, we can just replace the map */
6714 700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	612 if (view->index->map->hdr.messages_count <
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	613 ctx->finish_min_msg_count) {
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	614 mail_index_set_error(view->index,
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	615 "Index %s lost messages without expunging "
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	616 "(%u -> %u)", view->index->filepath,
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	617 view->map->hdr.messages_count,
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	618 view->index->map->hdr.messages_count);
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	619 ctx->finish_min_msg_count = 0;
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	620 view->inconsistent = TRUE;
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	621 }
700555e06849 If messages are unexpectedly lost from index file, log an error instead of Timo Sirainen <tss@iki.fi> parents: 6693 diff changeset	622
7623 1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	623 view->index->map->refcount++;
1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	624 mail_index_unmap(&view->map);
1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	625 view->map = view->index->map;
2044 d2eafac70369 View's message count shouldn't change until after view is synced. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	626 } else {
20818 139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	627 /* a) expunges seen. b) doing a partial sync because we saw
139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	628 a reset.
139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	629
139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	630 Create a private map which we update. If we're syncing
139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	631 expunges the map will finally be replaced with the head map
139b1e3beb85 lib-index: Fixed potential assert-crash when view syncing sees a reset marker. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20428 diff changeset	632 to remove the expunged messages. */
7623 1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	633 ctx->sync_map_update = TRUE;
5020 ac2be98ea811 unreliable_flags wasn't always set correctly, which caused "xxx counter Timo Sirainen <tss@iki.fi> parents: 5004 diff changeset	634
5769 901a9ba8cff7 View's counters can no longer be unreliable or broken. Timo Sirainen <tss@iki.fi> parents: 5768 diff changeset	635 if (view->map->refcount > 1) {
11039 0f98525e4567 Removed dead code. Timo Sirainen <tss@iki.fi> parents: 11038 diff changeset	636 tmp_map = mail_index_map_clone(view->map);
5853 d9f319a9d6e0 mail_index_map now contains pointer to index, so all functions taking map Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	637 mail_index_unmap(&view->map);
11039 0f98525e4567 Removed dead code. Timo Sirainen <tss@iki.fi> parents: 11038 diff changeset	638 view->map = tmp_map;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2869 diff changeset	639 }
7623 1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	640
1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	641 if (sync_expunges) {
1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	642 ctx->sync_new_map = view->index->map;
1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	643 ctx->sync_new_map->refcount++;
1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	644 }
2044 d2eafac70369 View's message count shouldn't change until after view is synced. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	645 }
8131 32a045eaf6b6 Fixed crashes related to struct mail_index_map_modseq handling. Timo Sirainen <tss@iki.fi> parents: 7932 diff changeset	646 mail_index_sync_map_init(&ctx->sync_map_ctx, view,
32a045eaf6b6 Fixed crashes related to struct mail_index_map_modseq handling. Timo Sirainen <tss@iki.fi> parents: 7932 diff changeset	647 MAIL_INDEX_SYNC_HANDLER_VIEW);
2044 d2eafac70369 View's message count shouldn't change until after view is synced. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	648
5785 bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5772 diff changeset	649 #ifdef DEBUG
5855 fd6ff4d9cab1 Removed view->hdr. Use view->map->hdr directly. Timo Sirainen <tss@iki.fi> parents: 5854 diff changeset	650 mail_index_map_check(view->map);
5785 bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5772 diff changeset	651 #endif
7872 3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	652 return ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	653 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	654
5768 35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	655 static bool
5771 433ae8a32506 s/view_sync_area_find/view_sync_is_hidden/ Timo Sirainen <tss@iki.fi> parents: 5769 diff changeset	656 view_sync_is_hidden(struct mail_index_view *view, uint32_t seq, uoff_t offset)
5653 965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	657 {
10000 c610321584ca Use array_foreach() in some useful places. Timo Sirainen <tss@iki.fi>* parents: 9971 diff changeset	658 const struct mail_index_view_log_sync_area *sync;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	659
5768 35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	660 if (!array_is_created(&view->syncs_hidden))
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	661 return FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	662
10000 c610321584ca Use array_foreach() in some useful places. Timo Sirainen <tss@iki.fi>* parents: 9971 diff changeset	663 array_foreach(&view->syncs_hidden, sync) {
c610321584ca Use array_foreach() in some useful places. Timo Sirainen <tss@iki.fi>* parents: 9971 diff changeset	664 if (sync->log_file_offset <= offset &&
c610321584ca Use array_foreach() in some useful places. Timo Sirainen <tss@iki.fi>* parents: 9971 diff changeset	665 offset - sync->log_file_offset < sync->length &&
c610321584ca Use array_foreach() in some useful places. Timo Sirainen <tss@iki.fi>* parents: 9971 diff changeset	666 sync->log_file_seq == seq)
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	667 return TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	668 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	669
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	670 return FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	671 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	672
5744 bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	673 static bool
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	674 mail_index_view_sync_want(struct mail_index_view_sync_ctx *ctx,
5744 bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	675 const struct mail_transaction_header *hdr)
bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	676 {
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	677 struct mail_index_view *view = ctx->view;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	678 uint32_t seq;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	679 uoff_t offset, next_offset;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	680
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	681 mail_transaction_log_view_get_prev_pos(view->log_view, &seq, &offset);
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	682 next_offset = offset + sizeof(*hdr) + hdr->size;
5744 bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	683
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	684 if ((hdr->type & (MAIL_TRANSACTION_EXPUNGE \|
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	685 MAIL_TRANSACTION_EXPUNGE_GUID)) != 0 &&
5768 35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	686 (hdr->type & MAIL_TRANSACTION_EXTERNAL) != 0) {
35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	687 if ((ctx->flags & MAIL_INDEX_VIEW_SYNC_FLAG_NOEXPUNGES) != 0) {
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	688 i_assert(!LOG_IS_BEFORE(seq, offset,
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	689 view->log_file_expunge_seq,
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	690 view->log_file_expunge_offset));
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	691 if (!ctx->skipped_expunges) {
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	692 view->log_file_expunge_seq = seq;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	693 view->log_file_expunge_offset = offset;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	694 ctx->skipped_expunges = TRUE;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	695 }
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	696 return FALSE;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	697 }
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	698 if (LOG_IS_BEFORE(seq, offset, view->log_file_expunge_seq,
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	699 view->log_file_expunge_offset)) {
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	700 /* already synced */
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	701 return FALSE;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	702 }
5744 bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	703 }
bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	704
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	705 if (LOG_IS_BEFORE(seq, offset, view->log_file_head_seq,
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	706 view->log_file_head_offset)) {
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	707 /* already synced */
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	708 return FALSE;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	709 }
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	710
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	711 view->log_file_head_seq = seq;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	712 view->log_file_head_offset = next_offset;
5744 bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	713 return TRUE;
bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	714 }
bf9f97c09250 Assert-crashfixes to handling non-external expunges (broken by last changes) Timo Sirainen <tss@iki.fi> parents: 5743 diff changeset	715
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	716 static int
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	717 mail_index_view_sync_get_next_transaction(struct mail_index_view_sync_ctx *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	718 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	719 struct mail_transaction_log_view *log_view = ctx->view->log_view;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	720 struct mail_index_view *view = ctx->view;
5728 7aab5d99fb45 Removed sync_mask parameter from mail_transaction_log_view_set(). The user Timo Sirainen <tss@iki.fi> parents: 5727 diff changeset	721 const struct mail_transaction_header *hdr;
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	722 uint32_t seq;
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	723 uoff_t offset;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	724 int ret;
5728 7aab5d99fb45 Removed sync_mask parameter from mail_transaction_log_view_set(). The user Timo Sirainen <tss@iki.fi> parents: 5727 diff changeset	725 bool synced_to_map;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	726
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	727 do {
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	728 /* Get the next transaction from log. */
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	729 ret = mail_transaction_log_view_next(log_view, &ctx->hdr,
5728 7aab5d99fb45 Removed sync_mask parameter from mail_transaction_log_view_set(). The user Timo Sirainen <tss@iki.fi> parents: 5727 diff changeset	730 &ctx->data);
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	731 if (ret <= 0) {
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	732 if (ret < 0)
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	733 return -1;
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	734
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	735 ctx->hdr = NULL;
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	736 ctx->last_read = TRUE;
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	737 return 0;
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	738 }
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	739
5728 7aab5d99fb45 Removed sync_mask parameter from mail_transaction_log_view_set(). The user Timo Sirainen <tss@iki.fi> parents: 5727 diff changeset	740 hdr = ctx->hdr;
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	741 /* skip records we've already synced */
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	742 } while (!mail_index_view_sync_want(ctx, hdr));
5728 7aab5d99fb45 Removed sync_mask parameter from mail_transaction_log_view_set(). The user Timo Sirainen <tss@iki.fi> parents: 5727 diff changeset	743
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	744 mail_transaction_log_view_get_prev_pos(log_view, &seq, &offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	745
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	746 /* If we started from a map that we didn't create ourself,
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	747 some of the transactions may already be synced. at the end
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	748 of this view sync we'll update file_seq=0 so that this check
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	749 always becomes FALSE for subsequent syncs. */
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	750 synced_to_map = view->map->hdr.log_file_seq != 0 &&
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	751 LOG_IS_BEFORE(seq, offset, view->map->hdr.log_file_seq,
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	752 view->map->hdr.log_file_head_offset);
5271 239fe5057748 View syncing fixes. Timo Sirainen <tss@iki.fi> parents: 5270 diff changeset	753
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	754 /* Apply transaction to view's mapping if needed (meaning we
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	755 didn't just re-map the view to head mapping). */
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	756 if (ctx->sync_map_update && !synced_to_map) {
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	757 if ((hdr->type & (MAIL_TRANSACTION_EXPUNGE \|
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	758 MAIL_TRANSACTION_EXPUNGE_GUID)) == 0) {
13428 ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 12782 diff changeset	759 ret = mail_index_sync_record(&ctx->sync_map_ctx,
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 12782 diff changeset	760 hdr, ctx->data);
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9621 diff changeset	761 }
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	762 if (ret < 0)
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	763 return -1;
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	764 }
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	765
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	766 ctx->hidden = view_sync_is_hidden(view, seq, offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	767 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	768 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	769
5768 35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	770 static bool
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	771 mail_index_view_sync_get_rec(struct mail_index_view_sync_ctx *ctx,
3825 a983412e1976 Changed mail_index_view_sync_next() to return struct Timo Sirainen <tss@iki.fi> parents: 3824 diff changeset	772 struct mail_index_view_sync_rec *rec)
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	773 {
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	774 const struct mail_transaction_header *hdr = ctx->hdr;
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	775 const void *data = ctx->data;
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	776
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	777 switch (hdr->type & MAIL_TRANSACTION_TYPE_MASK) {
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	778 case MAIL_TRANSACTION_FLAG_UPDATE: {
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	779 const struct mail_transaction_flag_update *update =
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	780 CONST_PTR_OFFSET(data, ctx->data_offset);
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	781
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	782 /* data contains mail_transaction_flag_update[] */
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	783 for (;;) {
8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	784 ctx->data_offset += sizeof(*update);
7812 d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset. Timo Sirainen <tss@iki.fi> parents: 7626 diff changeset	785 if (!MAIL_TRANSACTION_FLAG_UPDATE_IS_INTERNAL(update))
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	786 break;
8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	787
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	788 /* skip internal flag changes */
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	789 if (ctx->data_offset == ctx->hdr->size)
15298 4308d1794328 lib-index: Fix for handling view syncing for already deleted transaction logs. Timo Sirainen <tss@iki.fi> parents: 14133 diff changeset	790 return FALSE;
3018 43685e7aa7b1 Flag updates got lost if there were internal flag updates in same Timo Sirainen <tss@iki.fi> parents: 3016 diff changeset	791
43685e7aa7b1 Flag updates got lost if there were internal flag updates in same Timo Sirainen <tss@iki.fi> parents: 3016 diff changeset	792 update = CONST_PTR_OFFSET(data, ctx->data_offset);
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	793 }
3825 a983412e1976 Changed mail_index_view_sync_next() to return struct Timo Sirainen <tss@iki.fi> parents: 3824 diff changeset	794
15271 0f16da39121a Treat modseq updates explicitly instead of as if they were flag changes. Timo Sirainen <tss@iki.fi> parents: 14682 diff changeset	795 if (update->add_flags != 0 \|\| update->remove_flags != 0)
0f16da39121a Treat modseq updates explicitly instead of as if they were flag changes. Timo Sirainen <tss@iki.fi> parents: 14682 diff changeset	796 rec->type = MAIL_INDEX_VIEW_SYNC_TYPE_FLAGS;
0f16da39121a Treat modseq updates explicitly instead of as if they were flag changes. Timo Sirainen <tss@iki.fi> parents: 14682 diff changeset	797 else
0f16da39121a Treat modseq updates explicitly instead of as if they were flag changes. Timo Sirainen <tss@iki.fi> parents: 14682 diff changeset	798 rec->type = MAIL_INDEX_VIEW_SYNC_TYPE_MODSEQ;
3825 a983412e1976 Changed mail_index_view_sync_next() to return struct Timo Sirainen <tss@iki.fi> parents: 3824 diff changeset	799 rec->uid1 = update->uid1;
a983412e1976 Changed mail_index_view_sync_next() to return struct Timo Sirainen <tss@iki.fi> parents: 3824 diff changeset	800 rec->uid2 = update->uid2;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	801 break;
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	802 }
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	803 case MAIL_TRANSACTION_KEYWORD_UPDATE: {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	804 const struct mail_transaction_keyword_update *update = data;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	805 const uint32_t *uids;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	806
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	807 /* data contains mail_transaction_keyword_update header,
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	808 the keyword name and an array of { uint32_t uid1, uid2; } */
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	809
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	810 if (ctx->data_offset == 0) {
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	811 /* skip over the header and name */
3138 fbe844061fe2 Internal changes in how keywords are handled. struct mail_keywords isn't Timo Sirainen <tss@iki.fi> parents: 3018 diff changeset	812 ctx->data_offset = sizeof(*update) + update->name_size;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	813 if ((ctx->data_offset % 4) != 0)
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	814 ctx->data_offset += 4 - (ctx->data_offset % 4);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	815 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	816
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	817 uids = CONST_PTR_OFFSET(data, ctx->data_offset);
7875 8ded6c06bcfc Changed mail index view syncing API to return only flag changes. Timo Sirainen <tss@iki.fi> parents: 7874 diff changeset	818 rec->type = MAIL_INDEX_VIEW_SYNC_TYPE_FLAGS;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	819 rec->uid1 = uids[0];
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	820 rec->uid2 = uids[1];
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	821
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	822 ctx->data_offset += sizeof(uint32_t) * 2;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	823 break;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	824 }
3203 391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	825 case MAIL_TRANSACTION_KEYWORD_RESET: {
391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	826 const struct mail_transaction_keyword_reset *reset =
391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	827 CONST_PTR_OFFSET(data, ctx->data_offset);
391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	828
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	829 /* data contains mail_transaction_keyword_reset[] */
7875 8ded6c06bcfc Changed mail index view syncing API to return only flag changes. Timo Sirainen <tss@iki.fi> parents: 7874 diff changeset	830 rec->type = MAIL_INDEX_VIEW_SYNC_TYPE_FLAGS;
3203 391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	831 rec->uid1 = reset->uid1;
391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	832 rec->uid2 = reset->uid2;
391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	833 ctx->data_offset += sizeof(*reset);
391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	834 break;
391a1064872e mail_index_sync_next() now returns keyword updates (now only thing left to Timo Sirainen <tss@iki.fi> parents: 3187 diff changeset	835 }
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	836 default:
5768 35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	837 ctx->hdr = NULL;
35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	838 return FALSE;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	839 }
7624 271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	840
271131a1e060 View sync returns now also hidden records, but they're marked as such. Timo Sirainen <tss@iki.fi> parents: 7623 diff changeset	841 rec->hidden = ctx->hidden;
5768 35806800d2bd Removed more unneeded code. Timo Sirainen <tss@iki.fi> parents: 5767 diff changeset	842 return TRUE;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	843 }
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	844
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	845 static bool
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	846 mail_index_view_sync_next_lost(struct mail_index_view_sync_ctx *ctx,
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	847 struct mail_index_view_sync_rec *sync_rec)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	848 {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	849 const struct seq_range *range;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	850 unsigned int count;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	851
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	852 range = array_get(&ctx->lost_flags, &count);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	853 if (ctx->lost_flag_idx == count) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	854 ctx->last_read = TRUE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	855 return FALSE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	856 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	857
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	858 sync_rec->type = MAIL_INDEX_VIEW_SYNC_TYPE_FLAGS;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	859 sync_rec->uid1 = range[ctx->lost_flag_idx].seq1;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	860 sync_rec->uid2 = range[ctx->lost_flag_idx].seq2;
15298 4308d1794328 lib-index: Fix for handling view syncing for already deleted transaction logs. Timo Sirainen <tss@iki.fi> parents: 14133 diff changeset	861 sync_rec->hidden = FALSE;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	862 ctx->lost_flag_idx++;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	863 return TRUE;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	864 }
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	865
6279 f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	866 bool mail_index_view_sync_next(struct mail_index_view_sync_ctx *ctx,
f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	867 struct mail_index_view_sync_rec *sync_rec)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	868 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	869 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	870
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	871 if (ctx->log_was_lost)
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	872 return mail_index_view_sync_next_lost(ctx, sync_rec);
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	873
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	874 do {
8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	875 if (ctx->hdr == NULL \|\| ctx->data_offset == ctx->hdr->size) {
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	876 ret = mail_index_view_sync_get_next_transaction(ctx);
6279 f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	877 if (ret <= 0) {
f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	878 if (ret < 0)
f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	879 ctx->failed = TRUE;
f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	880 return FALSE;
f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	881 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	882
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	883 ctx->data_offset = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	884 }
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	885 } while (!mail_index_view_sync_get_rec(ctx, sync_rec));
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	886
6279 f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	887 return TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	888 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	889
4688 764bd841e0ae Try to avoid sending duplicate/useless flag updates. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	890 void mail_index_view_sync_get_expunges(struct mail_index_view_sync_ctx *ctx,
764bd841e0ae Try to avoid sending duplicate/useless flag updates. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	891 const ARRAY_TYPE(seq_range) **expunges_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	892 {
4688 764bd841e0ae Try to avoid sending duplicate/useless flag updates. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	893 *expunges_r = &ctx->expunges;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	894 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	895
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	896 static void
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	897 mail_index_view_sync_clean_log_syncs(struct mail_index_view *view)
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	898 {
5653 965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	899 const struct mail_index_view_log_sync_area *syncs;
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	900 unsigned int i, count;
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	901
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	902 if (!array_is_created(&view->syncs_hidden))
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	903 return;
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	904
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	905 /* Clean up to view's tail */
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	906 syncs = array_get(&view->syncs_hidden, &count);
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	907 for (i = 0; i < count; i++) {
5767 98a3ab6f8404 Removed no-appends specific code that's unneeded now. Timo Sirainen <tss@iki.fi> parents: 5764 diff changeset	908 if ((syncs[i].log_file_offset +
98a3ab6f8404 Removed no-appends specific code that's unneeded now. Timo Sirainen <tss@iki.fi> parents: 5764 diff changeset	909 syncs[i].length > view->log_file_expunge_offset &&
98a3ab6f8404 Removed no-appends specific code that's unneeded now. Timo Sirainen <tss@iki.fi> parents: 5764 diff changeset	910 syncs[i].log_file_seq == view->log_file_expunge_seq) \|\|
98a3ab6f8404 Removed no-appends specific code that's unneeded now. Timo Sirainen <tss@iki.fi> parents: 5764 diff changeset	911 syncs[i].log_file_seq > view->log_file_expunge_seq)
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	912 break;
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	913 }
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	914 if (i > 0)
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	915 array_delete(&view->syncs_hidden, 0, i);
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	916 }
3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3823 diff changeset	917
7815 2d62129a709b QRESYNC: If MODSEQs were returned in FETCH replies but there are pending Timo Sirainen <tss@iki.fi> parents: 7812 diff changeset	918 int mail_index_view_sync_commit(struct mail_index_view_sync_ctx **_ctx,
2d62129a709b QRESYNC: If MODSEQs were returned in FETCH replies but there are pending Timo Sirainen <tss@iki.fi> parents: 7812 diff changeset	919 bool *delayed_expunges_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	920 {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	921 struct mail_index_view_sync_ctx ctx = _ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	922 struct mail_index_view *view = ctx->view;
6675 a4f475d415d6 Added MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT flag. Timo Sirainen <tss@iki.fi> parents: 6673 diff changeset	923 int ret = ctx->failed ? -1 : 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	924
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	925 i_assert(view->syncing);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	926
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	927 *_ctx = NULL;
7815 2d62129a709b QRESYNC: If MODSEQs were returned in FETCH replies but there are pending Timo Sirainen <tss@iki.fi> parents: 7812 diff changeset	928 *delayed_expunges_r = ctx->skipped_expunges;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	929
6675 a4f475d415d6 Added MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT flag. Timo Sirainen <tss@iki.fi> parents: 6673 diff changeset	930 if ((!ctx->last_read \|\| view->inconsistent) &&
a4f475d415d6 Added MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT flag. Timo Sirainen <tss@iki.fi> parents: 6673 diff changeset	931 (ctx->flags & MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT) == 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	932 /* we didn't sync everything */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	933 view->inconsistent = TRUE;
6279 f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	934 ret = -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	935 }
8806 3cebcc8a3f11 indexes: Crashfix to error handling. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	936 if (ctx->sync_map_ctx.modseq_ctx != NULL)
7872 3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	937 mail_index_modseq_sync_end(&ctx->sync_map_ctx.modseq_ctx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	938
7623 1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	939 if (ctx->sync_new_map != NULL) {
5853 d9f319a9d6e0 mail_index_map now contains pointer to index, so all functions taking map Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	940 mail_index_unmap(&view->map);
7623 1285aedb353c Keep view's map always at least up-to-date with syncing. If there have been Timo Sirainen <tss@iki.fi> parents: 7620 diff changeset	941 view->map = ctx->sync_new_map;
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	942 } else if (ctx->sync_map_update) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	943 /* log offsets have no meaning in views. make sure they're not
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	944 tried to be used wrong by setting them to zero. */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	945 view->map->hdr.log_file_seq = 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	946 view->map->hdr.log_file_head_offset = 0;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	947 view->map->hdr.log_file_tail_offset = 0;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2044 diff changeset	948 }
5271 239fe5057748 View syncing fixes. Timo Sirainen <tss@iki.fi> parents: 5270 diff changeset	949
5925 2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	950 i_assert(view->map->hdr.messages_count >= ctx->finish_min_msg_count);
2b6909d5de3e We sometimes lost expunged messages. Added asserts to make sure it doesn't Timo Sirainen <tss@iki.fi> parents: 5867 diff changeset	951
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	952 if (!ctx->skipped_expunges) {
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	953 view->log_file_expunge_seq = view->log_file_head_seq;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	954 view->log_file_expunge_offset = view->log_file_head_offset;
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	955 }
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	956
7872 3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	957 if (ctx->sync_map_ctx.view != NULL)
3882c6cbfa99 mail_index_view_sync_begin() now delays its failures to _commit(). Timo Sirainen <tss@iki.fi> parents: 7815 diff changeset	958 mail_index_sync_map_deinit(&ctx->sync_map_ctx);
5747 c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	959 mail_index_view_sync_clean_log_syncs(ctx->view);
c71f611bbffc Removed syncs_done array. Instead track appends and expunges separately. Timo Sirainen <tss@iki.fi> parents: 5744 diff changeset	960
5065 fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5020 diff changeset	961 #ifdef DEBUG
5855 fd6ff4d9cab1 Removed view->hdr. Use view->map->hdr directly. Timo Sirainen <tss@iki.fi> parents: 5854 diff changeset	962 mail_index_map_check(view->map);
5065 fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5020 diff changeset	963 #endif
fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5020 diff changeset	964
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3825 diff changeset	965 /* set log view to empty range so unneeded memory gets freed */
6458 cca479cb3ef0 Keep transaction logs referenced while we know we need them. This fixes Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	966 mail_transaction_log_view_clear(view->log_view,
cca479cb3ef0 Keep transaction logs referenced while we know we need them. This fixes Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	967 view->log_file_expunge_seq);
1941 ced8e41c137a automatically fix broken/missing transaction log files on the fly Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	968
3346 8e093a2cc8db Expunge code cleanups. Timo Sirainen <tss@iki.fi> parents: 3278 diff changeset	969 if (array_is_created(&ctx->expunges))
8e093a2cc8db Expunge code cleanups. Timo Sirainen <tss@iki.fi> parents: 3278 diff changeset	970 array_free(&ctx->expunges);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	971 if (array_is_created(&ctx->lost_flags))
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7876 diff changeset	972 array_free(&ctx->lost_flags);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	973
7932 91758686277a View syncing: Keep track of highest modseq in views. If we lose transaction Timo Sirainen <tss@iki.fi> parents: 7885 diff changeset	974 view->highest_modseq = mail_index_map_modseq_get_highest(view->map);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	975 view->syncing = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	976 i_free(ctx);
6279 f52e7d1402b5 mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	977 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	978 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	979
4358 414cd456e7de We used "already synced log positions in view" array to store locations of Timo Sirainen <tss@iki.fi> parents: 4244 diff changeset	980 void mail_index_view_add_hidden_transaction(struct mail_index_view *view,
414cd456e7de We used "already synced log positions in view" array to store locations of Timo Sirainen <tss@iki.fi> parents: 4244 diff changeset	981 uint32_t log_file_seq,
5653 965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	982 uoff_t log_file_offset,
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	983 unsigned int length)
4358 414cd456e7de We used "already synced log positions in view" array to store locations of Timo Sirainen <tss@iki.fi> parents: 4244 diff changeset	984 {
5653 965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	985 struct mail_index_view_log_sync_area *area;
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	986
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	987 if (!array_is_created(&view->syncs_hidden))
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	988 i_array_init(&view->syncs_hidden, 32);
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	989
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	990 area = array_append_space(&view->syncs_hidden);
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	991 area->log_file_seq = log_file_seq;
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	992 area->log_file_offset = log_file_offset;
965e401fcb7b mail_index_view_add_hidden_transaction() takes now a whole area parameter so Timo Sirainen <tss@iki.fi> parents: 5303 diff changeset	993 area->length = length;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	994 }

Mercurial > dovecot > core-2.2

annotate src/lib-index/mail-index-view-sync.c @ 22576:707ae9de3812