dovecot/core-2.2: src/lib-index/mail-index-sync-update.c annotate

annotate src/lib-index/mail-index-sync-update.c @ 22576:707ae9de3812

lib: istream-multiplex - Minor code cleanup Avoid propagating the error twice, and avoid any confusion about what "got" actually contains.

author	Timo Sirainen <timo.sirainen@dovecot.fi>
date	Thu, 05 Oct 2017 20:24:11 +0300
parents	e0f6374d1209
children	bf758ce6ee6d

rev	line source
21390 2e2563132d5f Updated copyright notices to include the year 2017. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21389 diff changeset	1 /* Copyright (c) 2004-2017 Dovecot authors, see the included COPYING file */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	4 #include "ioloop.h"
3470 346a494c2feb Moved array declaration to array-decl.h and include it in lib.h. So array.h Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	5 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	6 #include "mmap-util.h"
7620 4b8c1c164d8f Initial CONDSTORE support. Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	7 #include "mail-index-modseq.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	8 #include "mail-index-view-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	9 #include "mail-index-sync-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	10 #include "mail-transaction-log.h"
3295 98923db29e39 If we're updating only log_file_ext_offset but log file sequence has Timo Sirainen <tss@iki.fi> parents: 3191 diff changeset	11 #include "mail-transaction-log-private.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	12
5851 fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	13 /* If we have less than this many bytes to sync from log file, don't bother
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	14 reading the main index */
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	15 #define MAIL_INDEX_SYNC_MIN_READ_INDEX_SIZE 2048
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	16
5065 fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	17 static void
fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	18 mail_index_sync_update_log_offset(struct mail_index_sync_map_ctx *ctx,
5255 005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	19 struct mail_index_map *map, bool eol)
5065 fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	20 {
fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	21 uint32_t prev_seq;
fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	22 uoff_t prev_offset;
fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	23
fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	24 mail_transaction_log_view_get_prev_pos(ctx->view->log_view,
fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	25 &prev_seq, &prev_offset);
7883 bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7882 diff changeset	26 if (prev_seq == 0) {
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7882 diff changeset	27 /* handling lost changes in view syncing */
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7882 diff changeset	28 return;
bc5eca410ed3 Mailbox view sync: Figure out the changes by comparing old and new maps if Timo Sirainen <tss@iki.fi> parents: 7882 diff changeset	29 }
5065 fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	30
5777 a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	31 if (!eol) {
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	32 if (prev_offset == ctx->ext_intro_end_offset &&
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	33 prev_seq == ctx->ext_intro_seq) {
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	34 /* previous transaction was an extension introduction.
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	35 we probably came here from
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	36 mail_index_sync_ext_reset(). if there are any more
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	37 views which want to continue syncing it needs the
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	38 intro. so back up a bit more.
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	39
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	40 don't do this in case the last transaction in the
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	41 log is the extension intro, so we don't keep trying
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	42 to sync it over and over again. */
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	43 prev_offset = ctx->ext_intro_offset;
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	44 }
8290 4402563ad86e index files: Another try at fixing handling log files with different header sizes. Timo Sirainen <tss@iki.fi> parents: 8131 diff changeset	45 map->hdr.log_file_seq = prev_seq;
5777 a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	46 } else {
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	47 i_assert(ctx->view->index->log->head->hdr.file_seq == prev_seq);
6927 9a11ee1c7912 Previous tail_offset change broke when rotating transaction logs. Timo Sirainen <tss@iki.fi> parents: 6924 diff changeset	48 if (map->hdr.log_file_seq != prev_seq) {
9a11ee1c7912 Previous tail_offset change broke when rotating transaction logs. Timo Sirainen <tss@iki.fi> parents: 6924 diff changeset	49 map->hdr.log_file_seq = prev_seq;
9a11ee1c7912 Previous tail_offset change broke when rotating transaction logs. Timo Sirainen <tss@iki.fi> parents: 6924 diff changeset	50 map->hdr.log_file_tail_offset = 0;
9a11ee1c7912 Previous tail_offset change broke when rotating transaction logs. Timo Sirainen <tss@iki.fi> parents: 6924 diff changeset	51 }
5255 005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	52 }
5749 420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	53 map->hdr.log_file_head_offset = prev_offset;
5065 fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	54 }
fe88d9c80638 Some mmap_disable=no fixes. Some debug additions. Some generic index fixes. Timo Sirainen <tss@iki.fi> parents: 5062 diff changeset	55
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	56 static void mail_index_sync_replace_map(struct mail_index_sync_map_ctx *ctx,
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	57 struct mail_index_map *map)
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	58 {
2909 f4eaf629bea3 Don't mark map to be written into disk if we're syncing a view and not Timo Sirainen <tss@iki.fi> parents: 2906 diff changeset	59 struct mail_index_view *view = ctx->view;
2911 f2c2b17d7659 Changed mail_index_map_to_memory() to mail_index_map_clone(). Even if the Timo Sirainen <tss@iki.fi> parents: 2909 diff changeset	60
4284 ae0d876d98f5 Memory leak fix for last update and some other cleanups Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	61 i_assert(view->map != map);
ae0d876d98f5 Memory leak fix for last update and some other cleanups Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	62
5777 a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	63 mail_index_sync_update_log_offset(ctx, view->map, FALSE);
5853 d9f319a9d6e0 mail_index_map now contains pointer to index, so all functions taking map Timo Sirainen <tss@iki.fi> parents: 5851 diff changeset	64 mail_index_unmap(&view->map);
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	65 view->map = map;
2909 f4eaf629bea3 Don't mark map to be written into disk if we're syncing a view and not Timo Sirainen <tss@iki.fi> parents: 2906 diff changeset	66
5772 002aa9bbfcb2 Removed index->hdr. Use index->map->hdr instead. Timo Sirainen <tss@iki.fi> parents: 5769 diff changeset	67 if (ctx->type != MAIL_INDEX_SYNC_HANDLER_VIEW)
4283 22b54c007b91 We were updating index->map also when syncing views. Timo Sirainen <tss@iki.fi> parents: 4279 diff changeset	68 view->index->map = map;
8131 32a045eaf6b6 Fixed crashes related to struct mail_index_map_modseq handling. Timo Sirainen <tss@iki.fi> parents: 7935 diff changeset	69
32a045eaf6b6 Fixed crashes related to struct mail_index_map_modseq handling. Timo Sirainen <tss@iki.fi> parents: 7935 diff changeset	70 mail_index_modseq_sync_map_replaced(ctx->modseq_ctx);
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	71 }
4283 22b54c007b91 We were updating index->map also when syncing views. Timo Sirainen <tss@iki.fi> parents: 4279 diff changeset	72
12292 84a9dc26108f lib-index: Work around clang static analyzer's false positive "dead code" issue. Timo Sirainen <tss@iki.fi> parents: 11738 diff changeset	73 static struct mail_index_map *
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	74 mail_index_sync_move_to_private_memory(struct mail_index_sync_map_ctx *ctx)
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	75 {
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	76 struct mail_index_map *map = ctx->view->map;
4026 20c335412d7d Added some locking related asserts, comments and when replacing index file, open it Timo Sirainen <timo.sirainen@movial.fi> parents: 4020 diff changeset	77
5867 1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	78 if (map->refcount > 1) {
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	79 map = mail_index_map_clone(map);
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	80 mail_index_sync_replace_map(ctx, map);
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	81 }
5867 1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	82
1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	83 if (!MAIL_INDEX_MAP_IS_IN_MEMORY(ctx->view->map))
1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	84 mail_index_map_move_to_memory(ctx->view->map);
8131 32a045eaf6b6 Fixed crashes related to struct mail_index_map_modseq handling. Timo Sirainen <tss@iki.fi> parents: 7935 diff changeset	85 mail_index_modseq_sync_map_replaced(ctx->modseq_ctx);
12292 84a9dc26108f lib-index: Work around clang static analyzer's false positive "dead code" issue. Timo Sirainen <tss@iki.fi> parents: 11738 diff changeset	86 return map;
5867 1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	87 }
1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	88
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	89 struct mail_index_map *
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	90 mail_index_sync_get_atomic_map(struct mail_index_sync_map_ctx *ctx)
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	91 {
14682 d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this. Timo Sirainen <tss@iki.fi> parents: 14676 diff changeset	92 (void)mail_index_sync_move_to_private_memory(ctx);
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	93 mail_index_record_map_move_to_private(ctx->view->map);
8131 32a045eaf6b6 Fixed crashes related to struct mail_index_map_modseq handling. Timo Sirainen <tss@iki.fi> parents: 7935 diff changeset	94 mail_index_modseq_sync_map_replaced(ctx->modseq_ctx);
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	95 return ctx->view->map;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	96 }
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	97
4498 33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	98 static int
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	99 mail_index_header_update_counts(struct mail_index_header *hdr,
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	100 uint8_t old_flags, uint8_t new_flags,
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	101 const char **error_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	102 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	103 if (((old_flags ^ new_flags) & MAIL_SEEN) != 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	104 /* different seen-flag */
4498 33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	105 if ((old_flags & MAIL_SEEN) != 0) {
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	106 if (hdr->seen_messages_count == 0) {
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	107 *error_r = "Seen counter wrong";
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	108 return -1;
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	109 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	110 hdr->seen_messages_count--;
4498 33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	111 } else {
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	112 if (hdr->seen_messages_count >= hdr->messages_count) {
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	113 *error_r = "Seen counter wrong";
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	114 return -1;
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	115 }
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	116
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	117 if (++hdr->seen_messages_count == hdr->messages_count)
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	118 hdr->first_unseen_uid_lowwater = hdr->next_uid;
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	119 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	120 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	121
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	122 if (((old_flags ^ new_flags) & MAIL_DELETED) != 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	123 /* different deleted-flag */
4498 33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	124 if ((old_flags & MAIL_DELETED) == 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	125 hdr->deleted_messages_count++;
4498 33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	126 if (hdr->deleted_messages_count > hdr->messages_count) {
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	127 *error_r = "Deleted counter wrong";
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	128 return -1;
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	129 }
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	130 } else {
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	131 if (hdr->deleted_messages_count == 0 \|\|
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	132 hdr->deleted_messages_count > hdr->messages_count) {
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	133 *error_r = "Deleted counter wrong";
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	134 return -1;
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	135 }
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	136
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	137 if (--hdr->deleted_messages_count == 0)
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	138 hdr->first_deleted_uid_lowwater = hdr->next_uid;
33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	139 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	140 }
4498 33391d888ea7 If header counters got broken when updating internal views, say it clearly Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	141 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	142 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	143
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	144 static void
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	145 mail_index_sync_header_update_counts_all(struct mail_index_sync_map_ctx *ctx,
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	146 uint32_t uid,
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	147 uint8_t old_flags, uint8_t new_flags)
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	148 {
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	149 struct mail_index_map const maps;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	150 const char *error;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	151 unsigned int i, count;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	152
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	153 maps = array_get(&ctx->view->map->rec_map->maps, &count);
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	154 for (i = 0; i < count; i++) {
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	155 if (uid >= maps[i]->hdr.next_uid)
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	156 continue;
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	157
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	158 if (mail_index_header_update_counts(&maps[i]->hdr,
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	159 old_flags, new_flags,
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	160 &error) < 0)
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	161 mail_index_sync_set_corrupted(ctx, "%s", error);
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	162 }
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	163 }
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	164
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	165 static void
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	166 mail_index_sync_header_update_counts(struct mail_index_sync_map_ctx *ctx,
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	167 uint32_t uid, uint8_t old_flags,
17288 4e3a518f6fcd lib-index: Small code cleanup Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	168 uint8_t new_flags)
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	169 {
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	170 const char *error;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	171
17288 4e3a518f6fcd lib-index: Small code cleanup Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	172 if (uid >= ctx->view->map->hdr.next_uid) {
16645 d6290bd28253 lib-index: Replaced assert-crash with "index corrupted" error. Timo Sirainen <tss@iki.fi> parents: 16644 diff changeset	173 mail_index_sync_set_corrupted(ctx, "uid %u >= next_uid %u",
d6290bd28253 lib-index: Replaced assert-crash with "index corrupted" error. Timo Sirainen <tss@iki.fi> parents: 16644 diff changeset	174 uid, ctx->view->map->hdr.next_uid);
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	175 } else {
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	176 if (mail_index_header_update_counts(&ctx->view->map->hdr,
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	177 old_flags, new_flags,
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	178 &error) < 0)
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	179 mail_index_sync_set_corrupted(ctx, "%s", error);
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	180 }
4244 b9a7a9592f43 Index header's seen/recent/deleted counters weren't always correct after Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	181 }
b9a7a9592f43 Index header's seen/recent/deleted counters weren't always correct after Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	182
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	183 static void
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	184 mail_index_header_update_lowwaters(struct mail_index_sync_map_ctx *ctx,
6263 7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	185 uint32_t uid, enum mail_flags flags)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	186 {
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	187 struct mail_index_map const maps;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	188 unsigned int i, count;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	189
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	190 maps = array_get(&ctx->view->map->rec_map->maps, &count);
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	191 for (i = 0; i < count; i++) {
6263 7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	192 if ((flags & MAIL_SEEN) == 0 &&
7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	193 uid < maps[i]->hdr.first_unseen_uid_lowwater)
7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	194 maps[i]->hdr.first_unseen_uid_lowwater = uid;
7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	195 if ((flags & MAIL_DELETED) != 0 &&
7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	196 uid < maps[i]->hdr.first_deleted_uid_lowwater)
7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	197 maps[i]->hdr.first_deleted_uid_lowwater = uid;
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	198 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	199 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	200
7054 4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	201 static void
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	202 sync_expunge_call_handlers(struct mail_index_sync_map_ctx *ctx,
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	203 uint32_t seq1, uint32_t seq2)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	204 {
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	205 const struct mail_index_expunge_handler *eh;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	206 struct mail_index_record *rec;
7363 dc24431136b3 Expunge handler calling fixes. Timo Sirainen <tss@iki.fi> parents: 7236 diff changeset	207 uint32_t seq;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	208
10000 c610321584ca Use array_foreach() in some useful places. Timo Sirainen <tss@iki.fi>* parents: 9725 diff changeset	209 array_foreach(&ctx->expunge_handlers, eh) {
7363 dc24431136b3 Expunge handler calling fixes. Timo Sirainen <tss@iki.fi> parents: 7236 diff changeset	210 for (seq = seq1; seq <= seq2; seq++) {
17428 4b1117b5cd8e mail-index: trivial robomatic migration from _MAP_IDX to new helper Phil Carmody <phil@dovecot.fi> parents: 17426 diff changeset	211 rec = MAIL_INDEX_REC_AT_SEQ(ctx->view->map, seq);
7054 4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	212 /* FIXME: does expunge handler's return value matter?
4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	213 we probably shouldn't disallow expunges if the
4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	214 handler returns failure.. should it be just changed
4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	215 to return void? */
7363 dc24431136b3 Expunge handler calling fixes. Timo Sirainen <tss@iki.fi> parents: 7236 diff changeset	216 (void)eh->handler(ctx, seq,
7054 4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	217 PTR_OFFSET(rec, eh->record_offset),
4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	218 eh->sync_context, eh->context);
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	219 }
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2086 diff changeset	220 }
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	221 }
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	222
17420 211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	223 static bool
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	224 sync_expunge_handlers_init(struct mail_index_sync_map_ctx *ctx)
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	225 {
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	226 /* call expunge handlers only when syncing index file */
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	227 if (ctx->type != MAIL_INDEX_SYNC_HANDLER_FILE)
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	228 return FALSE;
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	229
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	230 if (!ctx->expunge_handlers_set)
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	231 mail_index_sync_init_expunge_handlers(ctx);
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	232
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	233 if (!array_is_created(&ctx->expunge_handlers))
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	234 return FALSE;
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	235 return TRUE;
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	236 }
211c9493aaa4 mail-index: move expunge handler init into helper function Phil Carmody <phil@dovecot.fi> parents: 17416 diff changeset	237
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	238 static void
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	239 sync_expunge_range(struct mail_index_sync_map_ctx ctx, const ARRAY_TYPE(seq_range) seqs)
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	240 {
17423 74a726930ca8 mail-index: hoist initialisation of mail index map out of expunge loop Phil Carmody <phil@dovecot.fi> parents: 17422 diff changeset	241 struct mail_index_map *map;
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	242 const struct seq_range *range;
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	243 unsigned int i, count;
17426 0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	244 uint32_t dest_seq1, prev_seq2, orig_rec_count;
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	245
18008 1886e0616ab5 lib-index: Don't assert-crash if expunge event handling finds that everything is already expunged. Timo Sirainen <tss@iki.fi> parents: 17430 diff changeset	246 range = array_get(seqs, &count);
1886e0616ab5 lib-index: Don't assert-crash if expunge event handling finds that everything is already expunged. Timo Sirainen <tss@iki.fi> parents: 17430 diff changeset	247 if (count == 0)
1886e0616ab5 lib-index: Don't assert-crash if expunge event handling finds that everything is already expunged. Timo Sirainen <tss@iki.fi> parents: 17430 diff changeset	248 return;
1886e0616ab5 lib-index: Don't assert-crash if expunge event handling finds that everything is already expunged. Timo Sirainen <tss@iki.fi> parents: 17430 diff changeset	249
17423 74a726930ca8 mail-index: hoist initialisation of mail index map out of expunge loop Phil Carmody <phil@dovecot.fi> parents: 17422 diff changeset	250 map = mail_index_sync_get_atomic_map(ctx);
74a726930ca8 mail-index: hoist initialisation of mail index map out of expunge loop Phil Carmody <phil@dovecot.fi> parents: 17422 diff changeset	251
17421 aa6e69727edb mail-index: call all expunge handlers first Phil Carmody <phil@dovecot.fi> parents: 17420 diff changeset	252 /* call the expunge handlers first */
aa6e69727edb mail-index: call all expunge handlers first Phil Carmody <phil@dovecot.fi> parents: 17420 diff changeset	253 if (sync_expunge_handlers_init(ctx)) {
aa6e69727edb mail-index: call all expunge handlers first Phil Carmody <phil@dovecot.fi> parents: 17420 diff changeset	254 for (i = 0; i < count; i++) {
aa6e69727edb mail-index: call all expunge handlers first Phil Carmody <phil@dovecot.fi> parents: 17420 diff changeset	255 sync_expunge_call_handlers(ctx,
aa6e69727edb mail-index: call all expunge handlers first Phil Carmody <phil@dovecot.fi> parents: 17420 diff changeset	256 range[i].seq1, range[i].seq2);
aa6e69727edb mail-index: call all expunge handlers first Phil Carmody <phil@dovecot.fi> parents: 17420 diff changeset	257 }
aa6e69727edb mail-index: call all expunge handlers first Phil Carmody <phil@dovecot.fi> parents: 17420 diff changeset	258 }
aa6e69727edb mail-index: call all expunge handlers first Phil Carmody <phil@dovecot.fi> parents: 17420 diff changeset	259
17425 1466daa29957 mail-index: trivial paranoia check in expunge loop Phil Carmody <phil@dovecot.fi> parents: 17424 diff changeset	260 prev_seq2 = 0;
17426 0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	261 dest_seq1 = 1;
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	262 orig_rec_count = map->rec_map->records_count;
17424 b4c630542488 mail-index: Preparatory HACK - reverse the loop order in the expunge Phil Carmody <phil@dovecot.fi> parents: 17423 diff changeset	263 for (i = 0; i < count; i++) {
b4c630542488 mail-index: Preparatory HACK - reverse the loop order in the expunge Phil Carmody <phil@dovecot.fi> parents: 17423 diff changeset	264 uint32_t seq1 = range[i].seq1;
b4c630542488 mail-index: Preparatory HACK - reverse the loop order in the expunge Phil Carmody <phil@dovecot.fi> parents: 17423 diff changeset	265 uint32_t seq2 = range[i].seq2;
17422 25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	266 struct mail_index_record *rec;
25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	267 uint32_t seq_count, seq;
25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	268
17425 1466daa29957 mail-index: trivial paranoia check in expunge loop Phil Carmody <phil@dovecot.fi> parents: 17424 diff changeset	269 i_assert(seq1 > prev_seq2);
1466daa29957 mail-index: trivial paranoia check in expunge loop Phil Carmody <phil@dovecot.fi> parents: 17424 diff changeset	270
17422 25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	271 for (seq = seq1; seq <= seq2; seq++) {
17428 4b1117b5cd8e mail-index: trivial robomatic migration from _MAP_IDX to new helper Phil Carmody <phil@dovecot.fi> parents: 17426 diff changeset	272 rec = MAIL_INDEX_REC_AT_SEQ(map, seq);
17422 25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	273 mail_index_sync_header_update_counts(ctx, rec->uid, rec->flags, 0);
25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	274 }
25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	275
17426 0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	276 if (prev_seq2+1 <= seq1-1) {
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	277 /* @UNSAFE: move (prev_seq2+1) .. (seq1-1) to its
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	278 final location in the map if necessary */
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	279 uint32_t move_count = (seq1-1) - (prev_seq2+1) + 1;
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	280 if (prev_seq2+1-1 != dest_seq1-1)
17428 4b1117b5cd8e mail-index: trivial robomatic migration from _MAP_IDX to new helper Phil Carmody <phil@dovecot.fi> parents: 17426 diff changeset	281 memmove(MAIL_INDEX_REC_AT_SEQ(map, dest_seq1),
4b1117b5cd8e mail-index: trivial robomatic migration from _MAP_IDX to new helper Phil Carmody <phil@dovecot.fi> parents: 17426 diff changeset	282 MAIL_INDEX_REC_AT_SEQ(map, prev_seq2+1),
17426 0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	283 move_count * map->hdr.record_size);
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	284 dest_seq1 += move_count;
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	285 }
17422 25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	286 seq_count = seq2 - seq1 + 1;
25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	287 map->rec_map->records_count -= seq_count;
25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	288 map->hdr.messages_count -= seq_count;
25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	289 mail_index_modseq_expunge(ctx->modseq_ctx, seq1, seq2);
17425 1466daa29957 mail-index: trivial paranoia check in expunge loop Phil Carmody <phil@dovecot.fi> parents: 17424 diff changeset	290 prev_seq2 = seq2;
17422 25ce8f635980 mail-index: explicitly inline contents of sync_expunge() Phil Carmody <phil@dovecot.fi> parents: 17421 diff changeset	291 }
17426 0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	292 /* Final stragglers */
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	293 if (orig_rec_count > prev_seq2) {
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	294 uint32_t final_move_count = orig_rec_count - prev_seq2;
17428 4b1117b5cd8e mail-index: trivial robomatic migration from _MAP_IDX to new helper Phil Carmody <phil@dovecot.fi> parents: 17426 diff changeset	295 memmove(MAIL_INDEX_REC_AT_SEQ(map, dest_seq1),
4b1117b5cd8e mail-index: trivial robomatic migration from _MAP_IDX to new helper Phil Carmody <phil@dovecot.fi> parents: 17426 diff changeset	296 MAIL_INDEX_REC_AT_SEQ(map, prev_seq2+1),
17426 0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	297 final_move_count * map->hdr.record_size);
0b653039d3b9 mail-index: optimise memmoves in expunge, only move each region once Phil Carmody <phil@dovecot.fi> parents: 17425 diff changeset	298 }
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	299 }
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	300
9681 cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	301 static void sync_append_record(struct mail_index_map map)
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	302 {
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	303 size_t append_pos;
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	304 void *ret;
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	305
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	306 append_pos = map->rec_map->records_count * map->hdr.record_size;
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	307 ret = buffer_get_space_unsafe(map->rec_map->buffer, append_pos,
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	308 map->hdr.record_size);
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	309 map->rec_map->records =
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	310 buffer_get_modifiable_data(map->rec_map->buffer, NULL);
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	311 return ret;
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	312 }
cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	313
10367 75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	314 static bool sync_update_ignored_change(struct mail_index_sync_map_ctx *ctx)
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	315 {
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	316 struct mail_index_transaction_commit_result *result =
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	317 ctx->view->index->sync_commit_result;
11738 0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	318 uint32_t prev_log_seq;
0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	319 uoff_t prev_log_offset, trans_start_offset, trans_end_offset;
10367 75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	320
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	321 if (result == NULL)
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	322 return FALSE;
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	323
11738 0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	324 /* we'll return TRUE if this modseq change was written within the
0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	325 transaction that was just committed */
10367 75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	326 mail_transaction_log_view_get_prev_pos(ctx->view->log_view,
11738 0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	327 &prev_log_seq, &prev_log_offset);
0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	328 if (prev_log_seq != result->log_file_seq)
10367 75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	329 return FALSE;
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	330
11738 0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	331 trans_end_offset = result->log_file_offset;
0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	332 trans_start_offset = trans_end_offset - result->commit_size;
0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	333 if (prev_log_offset < trans_start_offset \|\|
0ffb55c4d323 lib-index: Minor code cleanup / commenting. Timo Sirainen <tss@iki.fi> parents: 11665 diff changeset	334 prev_log_offset >= trans_end_offset)
10367 75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	335 return FALSE;
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	336
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	337 return TRUE;
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	338 }
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	339
9691 b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	340 static int
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	341 sync_modseq_update(struct mail_index_sync_map_ctx *ctx,
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	342 const struct mail_transaction_modseq_update *u,
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	343 unsigned int size)
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	344 {
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	345 struct mail_index_view *view = ctx->view;
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	346 const struct mail_transaction_modseq_update *end;
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	347 uint32_t seq;
22426 e0f6374d1209 lib-index: Remove tracking of mail_index_modseq_sync.highest_modseq Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21530 diff changeset	348 uint64_t min_modseq;
10367 75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	349 int ret;
9691 b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	350
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	351 end = CONST_PTR_OFFSET(u, size);
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	352 for (; u < end; u++) {
9725 1411f157ffdd lib-index: Added mail_index_update_highest_modseq(). Timo Sirainen <tss@iki.fi> parents: 9724 diff changeset	353 if (u->uid == 0)
1411f157ffdd lib-index: Added mail_index_update_highest_modseq(). Timo Sirainen <tss@iki.fi> parents: 9724 diff changeset	354 seq = 0;
1411f157ffdd lib-index: Added mail_index_update_highest_modseq(). Timo Sirainen <tss@iki.fi> parents: 9724 diff changeset	355 else if (!mail_index_lookup_seq(view, u->uid, &seq))
9691 b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	356 continue;
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	357
16709 4d297ab881ef lib-index: Fixed modseq updates when modseq was over 32bits long Timo Sirainen <tss@iki.fi> parents: 16645 diff changeset	358 min_modseq = ((uint64_t)u->modseq_high32 << 32) \|
9691 b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	359 u->modseq_low32;
10367 75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	360
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	361 ret = seq == 0 ? 1 :
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	362 mail_index_modseq_set(view, seq, min_modseq);
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	363 if (ret < 0) {
9691 b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	364 mail_index_sync_set_corrupted(ctx,
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	365 "modseqs updated before they were enabled");
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	366 return -1;
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	367 }
10367 75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	368 if (ret == 0 && sync_update_ignored_change(ctx))
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored. Timo Sirainen <tss@iki.fi> parents: 10000 diff changeset	369 view->index->sync_commit_result->ignored_modseq_changes++;
9691 b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	370 }
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	371 return 1;
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	372 }
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	373
2953 ce8ff2279fb2 Sync contexts are always of type struct mail_index_sync_map_ctx. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	374 static int sync_append(const struct mail_index_record *rec,
ce8ff2279fb2 Sync contexts are always of type struct mail_index_sync_map_ctx. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	375 struct mail_index_sync_map_ctx *ctx)
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	376 {
2285 afb81e31b989 fixing crashes from last changes Timo Sirainen <tss@iki.fi> parents: 2280 diff changeset	377 struct mail_index_view *view = ctx->view;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	378 struct mail_index_map *map = view->map;
7882 ab08272053c5 Added assert. Timo Sirainen <tss@iki.fi> parents: 7812 diff changeset	379 const struct mail_index_record *old_rec;
6263 7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	380 enum mail_flags new_flags;
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2210 diff changeset	381 void *dest;
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2210 diff changeset	382
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	383 if (rec->uid < map->hdr.next_uid) {
4279 ca5008de2921 If we're synchronizing a view (and not index itself) from transaction log, Timo Sirainen <tss@iki.fi> parents: 4246 diff changeset	384 mail_index_sync_set_corrupted(ctx,
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	385 "Append with UID %u, but next_uid = %u",
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	386 rec->uid, map->hdr.next_uid);
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	387 return -1;
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	388 }
8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	389
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	390 /* move to memory. the mapping is written when unlocking so we don't
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	391 waste time re-mmap()ing multiple times or waste space growing index
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	392 file too large */
12292 84a9dc26108f lib-index: Work around clang static analyzer's false positive "dead code" issue. Timo Sirainen <tss@iki.fi> parents: 11738 diff changeset	393 map = mail_index_sync_move_to_private_memory(ctx);
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	394
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	395 if (rec->uid <= map->rec_map->last_appended_uid) {
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	396 i_assert(map->hdr.messages_count < map->rec_map->records_count);
6263 7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	397 /* the flags may have changed since it was added to map.
7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	398 use the updated flags already, so flag counters won't get
7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	399 broken. */
7882 ab08272053c5 Added assert. Timo Sirainen <tss@iki.fi> parents: 7812 diff changeset	400 old_rec = MAIL_INDEX_MAP_IDX(map, map->hdr.messages_count);
ab08272053c5 Added assert. Timo Sirainen <tss@iki.fi> parents: 7812 diff changeset	401 i_assert(old_rec->uid == rec->uid);
ab08272053c5 Added assert. Timo Sirainen <tss@iki.fi> parents: 7812 diff changeset	402 new_flags = old_rec->flags;
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	403 } else {
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	404 /* don't rely on buffer->used being at the correct position.
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	405 at least expunges can move it */
9681 cf187692fcfe Added ability to change existing messages' UIDs with mail_[index_]update_uid(). Timo Sirainen <tss@iki.fi> parents: 9624 diff changeset	406 dest = sync_append_record(map);
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	407 memcpy(dest, rec, sizeof(*rec));
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	408 memset(PTR_OFFSET(dest, sizeof(*rec)), 0,
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	409 map->hdr.record_size - sizeof(*rec));
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	410 map->rec_map->records_count++;
606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	411 map->rec_map->last_appended_uid = rec->uid;
6263 7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	412 new_flags = rec->flags;
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	413
7620 4b8c1c164d8f Initial CONDSTORE support. Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	414 mail_index_modseq_append(ctx->modseq_ctx,
4b8c1c164d8f Initial CONDSTORE support. Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	415 map->rec_map->records_count);
6167 606f3d9f2769 Prevent copying record data when appending new messages. Timo Sirainen <tss@iki.fi> parents: 6166 diff changeset	416 }
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	417
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	418 map->hdr.messages_count++;
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	419 map->hdr.next_uid = rec->uid+1;
3805 5d090d03f2b5 Don't rewrite index file every time with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	420
21530 c337e4fe88e5 lib-index: Add MAIL_INDEX_OPEN_FLAG_NO_DIRTY Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	421 if ((new_flags & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0 &&
c337e4fe88e5 lib-index: Add MAIL_INDEX_OPEN_FLAG_NO_DIRTY Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	422 (view->index->flags & MAIL_INDEX_OPEN_FLAG_NO_DIRTY) == 0)
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	423 map->hdr.flags \|= MAIL_INDEX_HDR_FLAG_HAVE_DIRTY;
2664 f54b0f04c628 hdr_flag_have_dirty wasn't set if dirty flags were set in appends. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	424
6263 7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	425 mail_index_header_update_lowwaters(ctx, rec->uid, new_flags);
17288 4e3a518f6fcd lib-index: Small code cleanup Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	426 mail_index_sync_header_update_counts(ctx, rec->uid, 0, new_flags);
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	427 return 1;
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	428 }
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	429
7132 a1d1f7c9671e Reverted fix for dropping expunged messages from flag updates. It's more Timo Sirainen <tss@iki.fi> parents: 7130 diff changeset	430 static int sync_flag_update(const struct mail_transaction_flag_update *u,
a1d1f7c9671e Reverted fix for dropping expunged messages from flag updates. It's more Timo Sirainen <tss@iki.fi> parents: 7130 diff changeset	431 struct mail_index_sync_map_ctx *ctx)
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	432 {
2285 afb81e31b989 fixing crashes from last changes Timo Sirainen <tss@iki.fi> parents: 2280 diff changeset	433 struct mail_index_view *view = ctx->view;
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2086 diff changeset	434 struct mail_index_record *rec;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	435 uint8_t flag_mask, old_flags;
17430 d3564b417e77 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops from [seq1..seq2] Phil Carmody <phil@dovecot.fi> parents: 17429 diff changeset	436 uint32_t seq, seq1, seq2;
7132 a1d1f7c9671e Reverted fix for dropping expunged messages from flag updates. It's more Timo Sirainen <tss@iki.fi> parents: 7130 diff changeset	437
a1d1f7c9671e Reverted fix for dropping expunged messages from flag updates. It's more Timo Sirainen <tss@iki.fi> parents: 7130 diff changeset	438 if (!mail_index_lookup_seq_range(view, u->uid1, u->uid2, &seq1, &seq2))
a1d1f7c9671e Reverted fix for dropping expunged messages from flag updates. It's more Timo Sirainen <tss@iki.fi> parents: 7130 diff changeset	439 return 1;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	440
7812 d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset. Timo Sirainen <tss@iki.fi> parents: 7798 diff changeset	441 if (!MAIL_TRANSACTION_FLAG_UPDATE_IS_INTERNAL(u)) {
d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset. Timo Sirainen <tss@iki.fi> parents: 7798 diff changeset	442 mail_index_modseq_update_flags(ctx->modseq_ctx,
d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset. Timo Sirainen <tss@iki.fi> parents: 7798 diff changeset	443 u->add_flags \| u->remove_flags,
d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset. Timo Sirainen <tss@iki.fi> parents: 7798 diff changeset	444 seq1, seq2);
d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset. Timo Sirainen <tss@iki.fi> parents: 7798 diff changeset	445 }
3805 5d090d03f2b5 Don't rewrite index file every time with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	446
21530 c337e4fe88e5 lib-index: Add MAIL_INDEX_OPEN_FLAG_NO_DIRTY Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	447 if ((u->add_flags & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0 &&
c337e4fe88e5 lib-index: Add MAIL_INDEX_OPEN_FLAG_NO_DIRTY Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	448 (view->index->flags & MAIL_INDEX_OPEN_FLAG_NO_DIRTY) == 0)
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	449 view->map->hdr.flags \|= MAIL_INDEX_HDR_FLAG_HAVE_DIRTY;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	450
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	451 flag_mask = ~u->remove_flags;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	452
5062 0cdded0843ca cleanups Timo Sirainen <tss@iki.fi> parents: 5022 diff changeset	453 if (((u->add_flags \| u->remove_flags) &
6037 d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 5868 diff changeset	454 (MAIL_SEEN \| MAIL_DELETED)) == 0) {
5062 0cdded0843ca cleanups Timo Sirainen <tss@iki.fi> parents: 5022 diff changeset	455 /* we're not modifying any counted/lowwatered flags */
17430 d3564b417e77 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops from [seq1..seq2] Phil Carmody <phil@dovecot.fi> parents: 17429 diff changeset	456 for (seq = seq1; seq <= seq2; seq++) {
d3564b417e77 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops from [seq1..seq2] Phil Carmody <phil@dovecot.fi> parents: 17429 diff changeset	457 rec = MAIL_INDEX_REC_AT_SEQ(view->map, seq);
5093 8a274bc9498c Setting flags other than seen/deleted was broken. Timo Sirainen <tss@iki.fi> parents: 5065 diff changeset	458 rec->flags = (rec->flags & flag_mask) \| u->add_flags;
8a274bc9498c Setting flags other than seen/deleted was broken. Timo Sirainen <tss@iki.fi> parents: 5065 diff changeset	459 }
4244 b9a7a9592f43 Index header's seen/recent/deleted counters weren't always correct after Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	460 } else {
17430 d3564b417e77 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops from [seq1..seq2] Phil Carmody <phil@dovecot.fi> parents: 17429 diff changeset	461 for (seq = seq1; seq <= seq2; seq++) {
d3564b417e77 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops from [seq1..seq2] Phil Carmody <phil@dovecot.fi> parents: 17429 diff changeset	462 rec = MAIL_INDEX_REC_AT_SEQ(view->map, seq);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	463
4244 b9a7a9592f43 Index header's seen/recent/deleted counters weren't always correct after Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	464 old_flags = rec->flags;
b9a7a9592f43 Index header's seen/recent/deleted counters weren't always correct after Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	465 rec->flags = (rec->flags & flag_mask) \| u->add_flags;
b9a7a9592f43 Index header's seen/recent/deleted counters weren't always correct after Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	466
6263 7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	467 mail_index_header_update_lowwaters(ctx, rec->uid,
7ecdd2a996cf Don't break flag counters when appending to shared record maps. Timo Sirainen <tss@iki.fi> parents: 6167 diff changeset	468 rec->flags);
17288 4e3a518f6fcd lib-index: Small code cleanup Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	469 mail_index_sync_header_update_counts_all(ctx, rec->uid,
4e3a518f6fcd lib-index: Small code cleanup Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	470 old_flags,
4e3a518f6fcd lib-index: Small code cleanup Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	471 rec->flags);
4244 b9a7a9592f43 Index header's seen/recent/deleted counters weren't always correct after Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	472 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	473 }
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	474 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	475 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	476
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	477 static int sync_header_update(const struct mail_transaction_header_update *u,
2953 ce8ff2279fb2 Sync contexts are always of type struct mail_index_sync_map_ctx. Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	478 struct mail_index_sync_map_ctx *ctx)
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	479 {
7236 b0b2c620a2b5 Code cleanup for last commit Timo Sirainen <tss@iki.fi> parents: 7235 diff changeset	480 #define MAIL_INDEX_HEADER_UPDATE_FIELD_IN_RANGE(u, field) \
b0b2c620a2b5 Code cleanup for last commit Timo Sirainen <tss@iki.fi> parents: 7235 diff changeset	481 ((u)->offset <= offsetof(struct mail_index_header, field) && \
b0b2c620a2b5 Code cleanup for last commit Timo Sirainen <tss@iki.fi> parents: 7235 diff changeset	482 (u)->offset + (u)->size > offsetof(struct mail_index_header, field))
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	483 struct mail_index_map *map = ctx->view->map;
5755 a380a5dc9307 Don't update log_file_tail_offset in header until syncing is finished. Timo Sirainen <tss@iki.fi> parents: 5749 diff changeset	484 uint32_t orig_log_file_tail_offset = map->hdr.log_file_tail_offset;
7235 40b6d4113e52 Log an error if uid_validity field is updated unexpectedly. If next_uid Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	485 uint32_t orig_next_uid = map->hdr.next_uid;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	486
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	487 if (u->offset >= map->hdr.base_header_size \|\|
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	488 u->offset + u->size > map->hdr.base_header_size) {
4279 ca5008de2921 If we're synchronizing a view (and not index itself) from transaction log, Timo Sirainen <tss@iki.fi> parents: 4246 diff changeset	489 mail_index_sync_set_corrupted(ctx,
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	490 "Header update outside range: %u + %u > %u",
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	491 u->offset, u->size, map->hdr.base_header_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	492 return -1;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	493 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	494
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	495 buffer_write(map->hdr_copy_buf, u->offset, u + 1, u->size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	496 map->hdr_base = map->hdr_copy_buf->data;
20281 b961ec20493e lib-index: Added header-size asserts Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	497 i_assert(map->hdr_copy_buf->used == map->hdr.header_size);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	498
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	499 /* @UNSAFE */
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	500 if ((uint32_t)(u->offset + u->size) <= sizeof(map->hdr)) {
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	501 memcpy(PTR_OFFSET(&map->hdr, u->offset),
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	502 u + 1, u->size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	503 } else if (u->offset < sizeof(map->hdr)) {
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	504 memcpy(PTR_OFFSET(&map->hdr, u->offset),
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	505 u + 1, sizeof(map->hdr) - u->offset);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	506 }
5755 a380a5dc9307 Don't update log_file_tail_offset in header until syncing is finished. Timo Sirainen <tss@iki.fi> parents: 5749 diff changeset	507
7235 40b6d4113e52 Log an error if uid_validity field is updated unexpectedly. If next_uid Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	508 if (map->hdr.next_uid < orig_next_uid) {
16644 374e338bb3d6 lib-index: Silently ignore next_uid shrinking without treating them as errors. Timo Sirainen <tss@iki.fi> parents: 16532 diff changeset	509 /* next_uid update tried to shrink its value. this can happen
374e338bb3d6 lib-index: Silently ignore next_uid shrinking without treating them as errors. Timo Sirainen <tss@iki.fi> parents: 16532 diff changeset	510 in some race conditions with e.g. with dsync, so just
374e338bb3d6 lib-index: Silently ignore next_uid shrinking without treating them as errors. Timo Sirainen <tss@iki.fi> parents: 16532 diff changeset	511 silently ignore it. */
7235 40b6d4113e52 Log an error if uid_validity field is updated unexpectedly. If next_uid Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	512 map->hdr.next_uid = orig_next_uid;
40b6d4113e52 Log an error if uid_validity field is updated unexpectedly. If next_uid Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	513 }
40b6d4113e52 Log an error if uid_validity field is updated unexpectedly. If next_uid Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	514
5755 a380a5dc9307 Don't update log_file_tail_offset in header until syncing is finished. Timo Sirainen <tss@iki.fi> parents: 5749 diff changeset	515 /* the tail offset updates are intended for internal transaction
a380a5dc9307 Don't update log_file_tail_offset in header until syncing is finished. Timo Sirainen <tss@iki.fi> parents: 5749 diff changeset	516 log handling. we'll update the offset in the header only when
a380a5dc9307 Don't update log_file_tail_offset in header until syncing is finished. Timo Sirainen <tss@iki.fi> parents: 5749 diff changeset	517 the sync is finished. */
a380a5dc9307 Don't update log_file_tail_offset in header until syncing is finished. Timo Sirainen <tss@iki.fi> parents: 5749 diff changeset	518 map->hdr.log_file_tail_offset = orig_log_file_tail_offset;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	519 return 1;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	520 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	521
13428 ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	522 static int
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	523 mail_index_sync_record_real(struct mail_index_sync_map_ctx *ctx,
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	524 const struct mail_transaction_header *hdr,
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	525 const void *data)
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	526 {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	527 int ret = 0;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	528
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	529 switch (hdr->type & MAIL_TRANSACTION_TYPE_MASK) {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	530 case MAIL_TRANSACTION_APPEND: {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	531 const struct mail_index_record rec, end;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	532
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	533 end = CONST_PTR_OFFSET(data, hdr->size);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	534 for (rec = data; rec < end; rec++) {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	535 ret = sync_append(rec, ctx);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	536 if (ret <= 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	537 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	538 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	539 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	540 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	541 case MAIL_TRANSACTION_EXPUNGE:
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	542 case MAIL_TRANSACTION_EXPUNGE\|MAIL_TRANSACTION_EXPUNGE_PROT: {
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	543 const struct mail_transaction_expunge rec = data, end;
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	544 ARRAY_TYPE(seq_range) seqs;
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	545 uint32_t seq1, seq2;
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	546
5743 1fa32e040ea5 Treat non-external expunges as requests for expunging messages. If it's Timo Sirainen <tss@iki.fi> parents: 5729 diff changeset	547 if ((hdr->type & MAIL_TRANSACTION_EXTERNAL) == 0) {
1fa32e040ea5 Treat non-external expunges as requests for expunging messages. If it's Timo Sirainen <tss@iki.fi> parents: 5729 diff changeset	548 /* this is simply a request for expunge */
1fa32e040ea5 Treat non-external expunges as requests for expunging messages. If it's Timo Sirainen <tss@iki.fi> parents: 5729 diff changeset	549 break;
1fa32e040ea5 Treat non-external expunges as requests for expunging messages. If it's Timo Sirainen <tss@iki.fi> parents: 5729 diff changeset	550 }
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	551 t_array_init(&seqs, 64);
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	552 end = CONST_PTR_OFFSET(data, hdr->size);
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	553 for (; rec != end; rec++) {
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	554 if (mail_index_lookup_seq_range(ctx->view,
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	555 rec->uid1, rec->uid2, &seq1, &seq2))
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	556 seq_range_array_add_range(&seqs, seq1, seq2);
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	557 }
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	558 sync_expunge_range(ctx, &seqs);
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	559 break;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	560 }
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	561 case MAIL_TRANSACTION_EXPUNGE_GUID:
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	562 case MAIL_TRANSACTION_EXPUNGE_GUID\|MAIL_TRANSACTION_EXPUNGE_PROT: {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	563 const struct mail_transaction_expunge_guid rec = data, end;
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	564 ARRAY_TYPE(seq_range) seqs;
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	565 uint32_t seq;
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	566
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	567 if ((hdr->type & MAIL_TRANSACTION_EXTERNAL) == 0) {
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	568 /* this is simply a request for expunge */
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	569 break;
2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	570 }
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	571 t_array_init(&seqs, 64);
9624 2558ba736207 Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). Timo Sirainen <tss@iki.fi> parents: 9001 diff changeset	572 end = CONST_PTR_OFFSET(data, hdr->size);
12487 e12572ca4454 lib-index: Added a missing sanity check for expunge-guid record's uid. Timo Sirainen <tss@iki.fi> parents: 12292 diff changeset	573 for (; rec != end; rec++) {
14883 222c7d752519 lib-index: Removed duplicate ext-intro checking code. Timo Sirainen <tss@iki.fi> parents: 14682 diff changeset	574 i_assert(rec->uid != 0);
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	575
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	576 if (mail_index_lookup_seq(ctx->view, rec->uid, &seq))
6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	577 seq_range_array_add(&seqs, seq);
12487 e12572ca4454 lib-index: Added a missing sanity check for expunge-guid record's uid. Timo Sirainen <tss@iki.fi> parents: 12292 diff changeset	578 }
13266 ee151b7e0d5a lib-index: Optimized handling MAIL_TRANSACTION_EXPUNGE_GUID records in large index. Timo Sirainen <tss@iki.fi> parents: 12782 diff changeset	579
17416 6d2176c558af lib-index: Optimize removing large number of expunges. Timo Sirainen <tss@iki.fi> parents: 17288 diff changeset	580 sync_expunge_range(ctx, &seqs);
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	581 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	582 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	583 case MAIL_TRANSACTION_FLAG_UPDATE: {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	584 const struct mail_transaction_flag_update rec, end;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	585
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	586 end = CONST_PTR_OFFSET(data, hdr->size);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	587 for (rec = data; rec < end; rec++) {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	588 ret = sync_flag_update(rec, ctx);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	589 if (ret <= 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	590 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	591 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	592 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	593 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	594 case MAIL_TRANSACTION_HEADER_UPDATE: {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	595 const struct mail_transaction_header_update *rec;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	596 unsigned int i;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	597
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	598 for (i = 0; i < hdr->size; ) {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	599 rec = CONST_PTR_OFFSET(data, i);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	600 ret = sync_header_update(rec, ctx);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	601 if (ret <= 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	602 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	603
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	604 i += sizeof(*rec) + rec->size;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	605 if ((i % 4) != 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	606 i += 4 - (i % 4);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	607 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	608 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	609 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	610 case MAIL_TRANSACTION_EXT_INTRO: {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	611 const struct mail_transaction_ext_intro *rec = data;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	612 unsigned int i;
5255 005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	613 uint32_t prev_seq;
005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	614 uoff_t prev_offset;
005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	615
005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	616 mail_transaction_log_view_get_prev_pos(ctx->view->log_view,
005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	617 &prev_seq, &prev_offset);
005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	618 ctx->ext_intro_seq = prev_seq;
005d73928f8f Don't set log sync position right after extension introduction, unless it's Timo Sirainen <tss@iki.fi> parents: 5225 diff changeset	619 ctx->ext_intro_offset = prev_offset;
5777 a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	620 ctx->ext_intro_end_offset =
a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	621 prev_offset + hdr->size + sizeof(*hdr);
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	622
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	623 for (i = 0; i < hdr->size; ) {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	624 if (i + sizeof(*rec) > hdr->size) {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	625 /* should be just extra padding */
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	626 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	627 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	628
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	629 rec = CONST_PTR_OFFSET(data, i);
14883 222c7d752519 lib-index: Removed duplicate ext-intro checking code. Timo Sirainen <tss@iki.fi> parents: 14682 diff changeset	630 /* name_size checked by _log_view_next() */
222c7d752519 lib-index: Removed duplicate ext-intro checking code. Timo Sirainen <tss@iki.fi> parents: 14682 diff changeset	631 i_assert(i + sizeof(*rec) + rec->name_size <= hdr->size);
5718 8b481177965a Moved duplicated log record validation code to a single place. Timo Sirainen <tss@iki.fi> parents: 5689 diff changeset	632
2954 9dc7a7b06781 Moved extension syncing code to separate file. Timo Sirainen <tss@iki.fi> parents: 2953 diff changeset	633 ret = mail_index_sync_ext_intro(ctx, rec);
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	634 if (ret <= 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	635 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	636
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	637 i += sizeof(*rec) + rec->name_size;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	638 if ((i % 4) != 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	639 i += 4 - (i % 4);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	640 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	641 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	642 }
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	643 case MAIL_TRANSACTION_EXT_RESET: {
7563 6de1aed24ce5 Added mail_index_ext_reset_inc() to atomically increase extension's Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	644 struct mail_transaction_ext_reset rec;
5721 18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	645
7563 6de1aed24ce5 Added mail_index_ext_reset_inc() to atomically increase extension's Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	646 /* old versions have only new_reset_id */
6de1aed24ce5 Added mail_index_ext_reset_inc() to atomically increase extension's Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	647 if (hdr->size < sizeof(uint32_t)) {
5721 18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	648 mail_index_sync_set_corrupted(ctx,
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	649 "ext reset: invalid record size");
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	650 ret = -1;
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	651 break;
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	652 }
21389 59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(p)) with the new i_zero() macro. Stephan Bosch <stephan.bosch@dovecot.fi>* parents: 21223 diff changeset	653 i_zero(&rec);
7563 6de1aed24ce5 Added mail_index_ext_reset_inc() to atomically increase extension's Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	654 memcpy(&rec, data, I_MIN(hdr->size, sizeof(rec)));
6de1aed24ce5 Added mail_index_ext_reset_inc() to atomically increase extension's Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	655 ret = mail_index_sync_ext_reset(ctx, &rec);
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	656 break;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	657 }
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	658 case MAIL_TRANSACTION_EXT_HDR_UPDATE: {
11039 0f98525e4567 Removed dead code. Timo Sirainen <tss@iki.fi> parents: 10674 diff changeset	659 const struct mail_transaction_ext_hdr_update *rec;
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	660 unsigned int i;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	661
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	662 for (i = 0; i < hdr->size; ) {
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	663 rec = CONST_PTR_OFFSET(data, i);
5721 18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	664
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	665 if (i + sizeof(*rec) > hdr->size \|\|
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	666 i + sizeof(*rec) + rec->size > hdr->size) {
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	667 mail_index_sync_set_corrupted(ctx,
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	668 "ext hdr update: invalid record size");
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	669 ret = -1;
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	670 break;
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	671 }
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	672
10380 8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	673 ret = mail_index_sync_ext_hdr_update(ctx, rec->offset,
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	674 rec->size, rec + 1);
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	675 if (ret <= 0)
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	676 break;
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	677
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	678 i += sizeof(*rec) + rec->size;
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	679 if ((i % 4) != 0)
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	680 i += 4 - (i % 4);
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	681 }
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	682 break;
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	683 }
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	684 case MAIL_TRANSACTION_EXT_HDR_UPDATE32: {
11039 0f98525e4567 Removed dead code. Timo Sirainen <tss@iki.fi> parents: 10674 diff changeset	685 const struct mail_transaction_ext_hdr_update32 *rec;
10380 8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	686 unsigned int i;
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	687
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	688 for (i = 0; i < hdr->size; ) {
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	689 rec = CONST_PTR_OFFSET(data, i);
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	690
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	691 if (i + sizeof(*rec) > hdr->size \|\|
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	692 i + sizeof(*rec) + rec->size > hdr->size) {
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	693 mail_index_sync_set_corrupted(ctx,
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	694 "ext hdr update: invalid record size");
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	695 ret = -1;
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	696 break;
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	697 }
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	698
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	699 ret = mail_index_sync_ext_hdr_update(ctx, rec->offset,
8b3c802556a9 mail_index_update_header_ext(): Couldn't handle >=64k headers. Timo Sirainen <tss@iki.fi> parents: 10367 diff changeset	700 rec->size, rec + 1);
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	701 if (ret <= 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	702 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	703
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	704 i += sizeof(*rec) + rec->size;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	705 if ((i % 4) != 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	706 i += 4 - (i % 4);
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	707 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	708 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	709 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	710 case MAIL_TRANSACTION_EXT_REC_UPDATE: {
5721 18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	711 const struct mail_transaction_ext_rec_update *rec;
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	712 unsigned int i, record_size;
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	713
6348 ef1c7b2acc10 Only index extension indexes are now called "ext_id". For map extension Timo Sirainen <tss@iki.fi> parents: 6346 diff changeset	714 if (ctx->cur_ext_map_idx == (uint32_t)-1) {
4847 7f250cd01843 Don't break if extension record size isn't divisible with 32bit. Timo Sirainen <tss@iki.fi> parents: 4501 diff changeset	715 mail_index_sync_set_corrupted(ctx,
5012 f1431e82375d cleanup Timo Sirainen <tss@iki.fi> parents: 4977 diff changeset	716 "Extension record updated "
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	717 "without intro prefix");
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	718 ret = -1;
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	719 break;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	720 }
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	721
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	722 if (ctx->cur_ext_ignore) {
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	723 ret = 1;
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	724 break;
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	725 }
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	726
4847 7f250cd01843 Don't break if extension record size isn't divisible with 32bit. Timo Sirainen <tss@iki.fi> parents: 4501 diff changeset	727 /* the record is padded to 32bits in the transaction log */
20323 07f21d0fb517 lib-index: Fixes to handling resized records. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20281 diff changeset	728 record_size = (sizeof(*rec) + ctx->cur_ext_record_size + 3) & ~3;
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	729
5721 18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	730 for (i = 0; i < hdr->size; i += record_size) {
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	731 rec = CONST_PTR_OFFSET(data, i);
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	732
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	733 if (i + record_size > hdr->size) {
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	734 mail_index_sync_set_corrupted(ctx,
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	735 "ext rec update: invalid record size");
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	736 ret = -1;
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	737 break;
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	738 }
18d7a680ffae Added more corruption detection code. Timo Sirainen <tss@iki.fi> parents: 5718 diff changeset	739
2954 9dc7a7b06781 Moved extension syncing code to separate file. Timo Sirainen <tss@iki.fi> parents: 2953 diff changeset	740 ret = mail_index_sync_ext_rec_update(ctx, rec);
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	741 if (ret <= 0)
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	742 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	743 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	744 break;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	745 }
8929 9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	746 case MAIL_TRANSACTION_EXT_ATOMIC_INC: {
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	747 const struct mail_transaction_ext_atomic_inc rec, end;
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	748
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	749 if (ctx->cur_ext_map_idx == (uint32_t)-1) {
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	750 mail_index_sync_set_corrupted(ctx,
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	751 "Extension record updated "
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	752 "without intro prefix");
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	753 ret = -1;
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	754 break;
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	755 }
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	756
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	757 if (ctx->cur_ext_ignore) {
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	758 ret = 1;
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	759 break;
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	760 }
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	761
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	762 end = CONST_PTR_OFFSET(data, hdr->size);
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	763 for (rec = data; rec < end; rec++) {
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	764 ret = mail_index_sync_ext_atomic_inc(ctx, rec);
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	765 if (ret <= 0)
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	766 break;
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	767 }
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	768 break;
9c50e7303513 Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	769 }
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 3008 diff changeset	770 case MAIL_TRANSACTION_KEYWORD_UPDATE: {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 3008 diff changeset	771 const struct mail_transaction_keyword_update *rec = data;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 3008 diff changeset	772
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 3008 diff changeset	773 ret = mail_index_sync_keywords(ctx, hdr, rec);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 3008 diff changeset	774 break;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 3008 diff changeset	775 }
3138 fbe844061fe2 Internal changes in how keywords are handled. struct mail_keywords isn't Timo Sirainen <tss@iki.fi> parents: 3134 diff changeset	776 case MAIL_TRANSACTION_KEYWORD_RESET: {
fbe844061fe2 Internal changes in how keywords are handled. struct mail_keywords isn't Timo Sirainen <tss@iki.fi> parents: 3134 diff changeset	777 const struct mail_transaction_keyword_reset *rec = data;
fbe844061fe2 Internal changes in how keywords are handled. struct mail_keywords isn't Timo Sirainen <tss@iki.fi> parents: 3134 diff changeset	778
fbe844061fe2 Internal changes in how keywords are handled. struct mail_keywords isn't Timo Sirainen <tss@iki.fi> parents: 3134 diff changeset	779 ret = mail_index_sync_keywords_reset(ctx, hdr, rec);
fbe844061fe2 Internal changes in how keywords are handled. struct mail_keywords isn't Timo Sirainen <tss@iki.fi> parents: 3134 diff changeset	780 break;
fbe844061fe2 Internal changes in how keywords are handled. struct mail_keywords isn't Timo Sirainen <tss@iki.fi> parents: 3134 diff changeset	781 }
9691 b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	782 case MAIL_TRANSACTION_MODSEQ_UPDATE: {
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	783 const struct mail_transaction_modseq_update *rec = data;
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	784
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	785 ret = sync_modseq_update(ctx, rec, hdr->size);
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	786 break;
b09d9350a2d9 Added ability to specify message's minimum modseq value. Timo Sirainen <tss@iki.fi> parents: 9681 diff changeset	787 }
10655 fc0ac73f0b36 Added support for marking mailbox index deleted. Don't allow any changes after that. Timo Sirainen <tss@iki.fi> parents: 10582 diff changeset	788 case MAIL_TRANSACTION_INDEX_DELETED:
10673 6552652a9504 lib-index: Index deletion is now a request that gets finalized by index sync. Timo Sirainen <tss@iki.fi> parents: 10655 diff changeset	789 if ((hdr->type & MAIL_TRANSACTION_EXTERNAL) == 0) {
6552652a9504 lib-index: Index deletion is now a request that gets finalized by index sync. Timo Sirainen <tss@iki.fi> parents: 10655 diff changeset	790 /* next sync finishes the deletion */
6552652a9504 lib-index: Index deletion is now a request that gets finalized by index sync. Timo Sirainen <tss@iki.fi> parents: 10655 diff changeset	791 ctx->view->index->index_delete_requested = TRUE;
6552652a9504 lib-index: Index deletion is now a request that gets finalized by index sync. Timo Sirainen <tss@iki.fi> parents: 10655 diff changeset	792 } else {
10674 22354f505277 lib-index: Added support for undeleting a deleted index. Timo Sirainen <tss@iki.fi> parents: 10673 diff changeset	793 /* transaction log reading handles this */
10673 6552652a9504 lib-index: Index deletion is now a request that gets finalized by index sync. Timo Sirainen <tss@iki.fi> parents: 10655 diff changeset	794 }
10655 fc0ac73f0b36 Added support for marking mailbox index deleted. Don't allow any changes after that. Timo Sirainen <tss@iki.fi> parents: 10582 diff changeset	795 break;
10674 22354f505277 lib-index: Added support for undeleting a deleted index. Timo Sirainen <tss@iki.fi> parents: 10673 diff changeset	796 case MAIL_TRANSACTION_INDEX_UNDELETED:
22354f505277 lib-index: Added support for undeleting a deleted index. Timo Sirainen <tss@iki.fi> parents: 10673 diff changeset	797 ctx->view->index->index_delete_requested = FALSE;
22354f505277 lib-index: Added support for undeleting a deleted index. Timo Sirainen <tss@iki.fi> parents: 10673 diff changeset	798 break;
12493 b7d13ee51aa4 lib-index: Added "transaction boundary" record for future compatibility. Timo Sirainen <tss@iki.fi> parents: 12487 diff changeset	799 case MAIL_TRANSACTION_BOUNDARY:
b7d13ee51aa4 lib-index: Added "transaction boundary" record for future compatibility. Timo Sirainen <tss@iki.fi> parents: 12487 diff changeset	800 break;
16191 b2fad9b21e60 lib-index: Backported MAIL_TRANSACTION_ATTRIBUTE_UPDATE from v2.2 Timo Sirainen <tss@iki.fi> parents: 14133 diff changeset	801 case MAIL_TRANSACTION_ATTRIBUTE_UPDATE:
b2fad9b21e60 lib-index: Backported MAIL_TRANSACTION_ATTRIBUTE_UPDATE from v2.2 Timo Sirainen <tss@iki.fi> parents: 14133 diff changeset	802 break;
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	803 default:
6760 74f4cea90bcd If transaction log contains records with unknown types, log an error instead Timo Sirainen <tss@iki.fi> parents: 6677 diff changeset	804 mail_index_sync_set_corrupted(ctx,
74f4cea90bcd If transaction log contains records with unknown types, log an error instead Timo Sirainen <tss@iki.fi> parents: 6677 diff changeset	805 "Unknown transaction record type 0x%x",
74f4cea90bcd If transaction log contains records with unknown types, log an error instead Timo Sirainen <tss@iki.fi> parents: 6677 diff changeset	806 (hdr->type & MAIL_TRANSACTION_TYPE_MASK));
74f4cea90bcd If transaction log contains records with unknown types, log an error instead Timo Sirainen <tss@iki.fi> parents: 6677 diff changeset	807 ret = -1;
74f4cea90bcd If transaction log contains records with unknown types, log an error instead Timo Sirainen <tss@iki.fi> parents: 6677 diff changeset	808 break;
2762 a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	809 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	810 return ret;
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	811 }
a1c7e92c1839 New way of handling extension introductions in transaction log. Timo Sirainen <tss@iki.fi> parents: 2760 diff changeset	812
13428 ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	813 int mail_index_sync_record(struct mail_index_sync_map_ctx *ctx,
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	814 const struct mail_transaction_header *hdr,
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	815 const void *data)
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	816 {
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	817 int ret;
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	818
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	819 T_BEGIN {
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	820 ret = mail_index_sync_record_real(ctx, hdr, data);
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	821 } T_END;
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	822 return ret;
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	823 }
ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	824
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	825 void mail_index_sync_map_init(struct mail_index_sync_map_ctx *sync_map_ctx,
2869 d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	826 struct mail_index_view *view,
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	827 enum mail_index_sync_handler_type type)
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	828 {
21389 59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(p)) with the new i_zero() macro. Stephan Bosch <stephan.bosch@dovecot.fi>* parents: 21223 diff changeset	829 i_zero(sync_map_ctx);
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	830 sync_map_ctx->view = view;
6348 ef1c7b2acc10 Only index extension indexes are now called "ext_id". For map extension Timo Sirainen <tss@iki.fi> parents: 6346 diff changeset	831 sync_map_ctx->cur_ext_map_idx = (uint32_t)-1;
2869 d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	832 sync_map_ctx->type = type;
7620 4b8c1c164d8f Initial CONDSTORE support. Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	833 sync_map_ctx->modseq_ctx = mail_index_modseq_sync_begin(sync_map_ctx);
2869 d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	834
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	835 mail_index_sync_init_handlers(sync_map_ctx);
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	836 }
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	837
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	838 void mail_index_sync_map_deinit(struct mail_index_sync_map_ctx *sync_map_ctx)
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	839 {
7620 4b8c1c164d8f Initial CONDSTORE support. Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	840 i_assert(sync_map_ctx->modseq_ctx == NULL);
4b8c1c164d8f Initial CONDSTORE support. Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	841
6346 5dcd55d1a47f Complain about unknown extension introduction IDs in log only once per sync. Timo Sirainen <tss@iki.fi> parents: 6342 diff changeset	842 if (sync_map_ctx->unknown_extensions != NULL)
6414 a6a49d5efc59 Changed buffer_free() and buffer_free_without_data() APIs to take ** pointer Timo Sirainen <tss@iki.fi> parents: 6389 diff changeset	843 buffer_free(&sync_map_ctx->unknown_extensions);
2869 d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	844 if (sync_map_ctx->expunge_handlers_used)
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	845 mail_index_sync_deinit_expunge_handlers(sync_map_ctx);
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2861 diff changeset	846 mail_index_sync_deinit_handlers(sync_map_ctx);
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	847 }
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2819 diff changeset	848
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	849 static void mail_index_sync_update_hdr_dirty_flag(struct mail_index_map *map)
5278 d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5271 diff changeset	850 {
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	851 const struct mail_index_record *rec;
17429 293966cc6e39 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops over every record Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	852 uint32_t seq;
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	853
21530 c337e4fe88e5 lib-index: Add MAIL_INDEX_OPEN_FLAG_NO_DIRTY Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	854 if ((map->hdr.flags & MAIL_INDEX_HDR_FLAG_HAVE_DIRTY) != 0 \|\|
c337e4fe88e5 lib-index: Add MAIL_INDEX_OPEN_FLAG_NO_DIRTY Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21390 diff changeset	855 (map->index->flags & MAIL_INDEX_OPEN_FLAG_NO_DIRTY) != 0)
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	856 return;
5278 d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5271 diff changeset	857
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	858 /* do we have dirty flags anymore? */
17429 293966cc6e39 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops over every record Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	859 for (seq = 1; seq <= map->rec_map->records_count; seq++) {
293966cc6e39 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops over every record Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	860 rec = MAIL_INDEX_REC_AT_SEQ(map, seq);
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	861 if ((rec->flags & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0) {
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	862 map->hdr.flags \|= MAIL_INDEX_HDR_FLAG_HAVE_DIRTY;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	863 break;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	864 }
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	865 }
5278 d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5271 diff changeset	866 }
d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5271 diff changeset	867
5785 bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	868 #ifdef DEBUG
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	869 void mail_index_map_check(struct mail_index_map *map)
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	870 {
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	871 const struct mail_index_header *hdr = &map->hdr;
17429 293966cc6e39 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops over every record Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	872 unsigned int del = 0, seen = 0;
293966cc6e39 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops over every record Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	873 uint32_t seq, prev_uid = 0;
5785 bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	874
6166 004534216202 DEBUG: compile fix Timo Sirainen <tss@iki.fi> parents: 6165 diff changeset	875 i_assert(hdr->messages_count <= map->rec_map->records_count);
17429 293966cc6e39 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops over every record Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	876 for (seq = 1; seq <= hdr->messages_count; seq++) {
5785 bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	877 const struct mail_index_record *rec;
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	878
17429 293966cc6e39 mail-index: use _REC_AT_SEQ not _MAP_IDX in loops over every record Phil Carmody <phil@dovecot.fi> parents: 17428 diff changeset	879 rec = MAIL_INDEX_REC_AT_SEQ(map, seq);
7420 a4a552321bd3 DEBUG: Verify that UIDs are ascending after sync. Timo Sirainen <tss@iki.fi> parents: 7363 diff changeset	880 i_assert(rec->uid > prev_uid);
a4a552321bd3 DEBUG: Verify that UIDs are ascending after sync. Timo Sirainen <tss@iki.fi> parents: 7363 diff changeset	881 prev_uid = rec->uid;
5785 bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	882
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	883 if (rec->flags & MAIL_DELETED) {
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	884 i_assert(rec->uid >= hdr->first_deleted_uid_lowwater);
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	885 del++;
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	886 }
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	887 if (rec->flags & MAIL_SEEN)
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	888 seen++;
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	889 else
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	890 i_assert(rec->uid >= hdr->first_unseen_uid_lowwater);
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	891 }
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	892 i_assert(del == hdr->deleted_messages_count);
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	893 i_assert(seen == hdr->seen_messages_count);
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	894 }
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	895 #endif
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	896
5853 d9f319a9d6e0 mail_index_map now contains pointer to index, so all functions taking map Timo Sirainen <tss@iki.fi> parents: 5851 diff changeset	897 int mail_index_sync_map(struct mail_index_map **_map,
21087 3b909cd1264b lib-index: Improve error messages when transaction log is unexpectedly lost Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20337 diff changeset	898 enum mail_index_sync_handler_type type, bool force,
3b909cd1264b lib-index: Improve error messages when transaction log is unexpectedly lost Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20337 diff changeset	899 const char *sync_reason)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	900 {
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	901 struct mail_index_map map = _map;
5853 d9f319a9d6e0 mail_index_map now contains pointer to index, so all functions taking map Timo Sirainen <tss@iki.fi> parents: 5851 diff changeset	902 struct mail_index *index = map->index;
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	903 struct mail_index_view *view;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	904 struct mail_index_sync_map_ctx sync_map_ctx;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	905 const struct mail_transaction_header *thdr;
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	906 const void *tdata;
5847 55699bbeaec2 Removed unneeded code. Timo Sirainen <tss@iki.fi> parents: 5828 diff changeset	907 uint32_t prev_seq;
55699bbeaec2 Removed unneeded code. Timo Sirainen <tss@iki.fi> parents: 5828 diff changeset	908 uoff_t start_offset, prev_offset;
18627 69630e6048fd lib-index: If header is corrupted after syncing, log the reason why. Timo Sirainen <tss@iki.fi> parents: 18398 diff changeset	909 const char reason, error;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3806 diff changeset	910 int ret;
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	911 bool had_dirty, reset;
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	912
5773 622ab4de7d66 Keep index->map always usable, never NULL. Timo Sirainen <tss@iki.fi> parents: 5772 diff changeset	913 i_assert(index->map == map \|\| type == MAIL_INDEX_SYNC_HANDLER_VIEW);
622ab4de7d66 Keep index->map always usable, never NULL. Timo Sirainen <tss@iki.fi> parents: 5772 diff changeset	914
7143 81d7093fb892 If mmap_disable=yes, don't reopen index file unless we really have to, Timo Sirainen <tss@iki.fi> parents: 7132 diff changeset	915 if (index->log->head == NULL) {
81d7093fb892 If mmap_disable=yes, don't reopen index file unless we really have to, Timo Sirainen <tss@iki.fi> parents: 7132 diff changeset	916 i_assert(!force);
81d7093fb892 If mmap_disable=yes, don't reopen index file unless we really have to, Timo Sirainen <tss@iki.fi> parents: 7132 diff changeset	917 return 0;
81d7093fb892 If mmap_disable=yes, don't reopen index file unless we really have to, Timo Sirainen <tss@iki.fi> parents: 7132 diff changeset	918 }
81d7093fb892 If mmap_disable=yes, don't reopen index file unless we really have to, Timo Sirainen <tss@iki.fi> parents: 7132 diff changeset	919
7144 0734af67ca99 Fixed index reopen check to work more correctly when syncing view/head. Timo Sirainen <tss@iki.fi> parents: 7143 diff changeset	920 start_offset = type == MAIL_INDEX_SYNC_HANDLER_FILE ?
0734af67ca99 Fixed index reopen check to work more correctly when syncing view/head. Timo Sirainen <tss@iki.fi> parents: 7143 diff changeset	921 map->hdr.log_file_tail_offset : map->hdr.log_file_head_offset;
9696 126132cb1c19 index: Removed duplication of mail_index_open_flags from struct mail_index. Timo Sirainen <tss@iki.fi> parents: 9691 diff changeset	922 if (!force && (index->flags & MAIL_INDEX_OPEN_FLAG_MMAP_DISABLE) == 0) {
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	923 /* see if we'd prefer to reopen the index file instead of
7143 81d7093fb892 If mmap_disable=yes, don't reopen index file unless we really have to, Timo Sirainen <tss@iki.fi> parents: 7132 diff changeset	924 syncing the current map from the transaction log.
81d7093fb892 If mmap_disable=yes, don't reopen index file unless we really have to, Timo Sirainen <tss@iki.fi> parents: 7132 diff changeset	925 don't check this if mmap is disabled, because reopening
81d7093fb892 If mmap_disable=yes, don't reopen index file unless we really have to, Timo Sirainen <tss@iki.fi> parents: 7132 diff changeset	926 index causes sync to get lost. */
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	927 uoff_t log_size, index_size;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	928
5851 fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	929 if (index->fd == -1 &&
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	930 index->log->head->hdr.prev_file_seq != 0) {
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	931 /* we don't know the index's size, so use the
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	932 smallest index size we're willing to read */
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	933 index_size = MAIL_INDEX_SYNC_MIN_READ_INDEX_SIZE;
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	934 } else {
fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	935 index_size = map->hdr.header_size +
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	936 map->rec_map->records_count *
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	937 map->hdr.record_size;
5851 fb91e7d729d1 If we can do initial sync from .log file by reading less than 2k, do it Timo Sirainen <tss@iki.fi> parents: 5849 diff changeset	938 }
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	939
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	940 /* this isn't necessary correct currently, but it should be
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	941 close enough */
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	942 log_size = index->log->head->last_size;
7144 0734af67ca99 Fixed index reopen check to work more correctly when syncing view/head. Timo Sirainen <tss@iki.fi> parents: 7143 diff changeset	943 if (log_size > start_offset &&
0734af67ca99 Fixed index reopen check to work more correctly when syncing view/head. Timo Sirainen <tss@iki.fi> parents: 7143 diff changeset	944 log_size - start_offset > index_size)
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	945 return 0;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	946 }
2285 afb81e31b989 fixing crashes from last changes Timo Sirainen <tss@iki.fi> parents: 2280 diff changeset	947
5854 eca7e29dce0d Rewrite index lock handling. Timo Sirainen <tss@iki.fi> parents: 5853 diff changeset	948 view = mail_index_view_open_with_map(index, map);
5774 1af7024361e9 If a forced syncing fails, it could be because of broken log file position. Timo Sirainen <tss@iki.fi> parents: 5773 diff changeset	949 ret = mail_transaction_log_view_set(view->log_view,
1af7024361e9 If a forced syncing fails, it could be because of broken log file position. Timo Sirainen <tss@iki.fi> parents: 5773 diff changeset	950 map->hdr.log_file_seq, start_offset,
18398 8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	951 (uint32_t)-1, (uoff_t)-1,
8146fdc0de34 lib-index: Added reason_r parameter to mail_transaction_log_view_set() Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	952 &reset, &reason);
5774 1af7024361e9 If a forced syncing fails, it could be because of broken log file position. Timo Sirainen <tss@iki.fi> parents: 5773 diff changeset	953 if (ret <= 0) {
11363 b08abace44ed lib-index: On errors, close sync view before fsck to avoid assert-crash. Timo Sirainen <tss@iki.fi> parents: 11039 diff changeset	954 mail_index_view_close(&view);
20337 9ab47cf0fb26 lib-index: Don't break indexes on syscall failures during index refreshing. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20323 diff changeset	955 if (force && ret < 0) {
9ab47cf0fb26 lib-index: Don't break indexes on syscall failures during index refreshing. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20323 diff changeset	956 /* if we failed because of a syscall error, make sure
9ab47cf0fb26 lib-index: Don't break indexes on syscall failures during index refreshing. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20323 diff changeset	957 we return a failure. */
9ab47cf0fb26 lib-index: Don't break indexes on syscall failures during index refreshing. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20323 diff changeset	958 return -1;
9ab47cf0fb26 lib-index: Don't break indexes on syscall failures during index refreshing. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20323 diff changeset	959 }
5774 1af7024361e9 If a forced syncing fails, it could be because of broken log file position. Timo Sirainen <tss@iki.fi> parents: 5773 diff changeset	960 if (force && ret == 0) {
1af7024361e9 If a forced syncing fails, it could be because of broken log file position. Timo Sirainen <tss@iki.fi> parents: 5773 diff changeset	961 /* the seq/offset is probably broken */
11659 3156315704ef lib-index: Added a missing error logging. Timo Sirainen <tss@iki.fi> parents: 11363 diff changeset	962 mail_index_set_error(index, "Index %s: Lost log for "
21087 3b909cd1264b lib-index: Improve error messages when transaction log is unexpectedly lost Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20337 diff changeset	963 "seq=%u offset=%"PRIuUOFF_T": %s "
3b909cd1264b lib-index: Improve error messages when transaction log is unexpectedly lost Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20337 diff changeset	964 "(initial_mapped=%d, reason=%s)", index->filepath,
3b909cd1264b lib-index: Improve error messages when transaction log is unexpectedly lost Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20337 diff changeset	965 map->hdr.log_file_seq, start_offset, reason,
21112 bd1fdf504ccc Fix -Wstrict-bool compiler warnings. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21087 diff changeset	966 index->initial_mapped ? 1 : 0, sync_reason);
5774 1af7024361e9 If a forced syncing fails, it could be because of broken log file position. Timo Sirainen <tss@iki.fi> parents: 5773 diff changeset	967 (void)mail_index_fsck(index);
1af7024361e9 If a forced syncing fails, it could be because of broken log file position. Timo Sirainen <tss@iki.fi> parents: 5773 diff changeset	968 }
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	969 /* can't use it. sync by re-reading index. */
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	970 return 0;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	971 }
1922 e4c30cb5d094 Locking changes. bugfixes. Timo Sirainen <tss@iki.fi> parents: 1919 diff changeset	972
8993 48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	973 mail_transaction_log_get_head(index->log, &prev_seq, &prev_offset);
48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	974 if (prev_seq != map->hdr.log_file_seq \|\|
48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	975 prev_offset - map->hdr.log_file_tail_offset >
48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	976 MAIL_INDEX_MIN_WRITE_BYTES) {
48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	977 /* we're reading more from log than we would have preferred.
48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	978 remember that we probably want to rewrite index soon. */
48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	979 index->index_min_write = TRUE;
48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	980 }
48d2377939c2 Write to main index file less often. Timo Sirainen <tss@iki.fi> parents: 8929 diff changeset	981
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	982 /* view referenced the map. avoid unnecessary map cloning by
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	983 unreferencing the map while view exists. */
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	984 map->refcount--;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	985
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	986 had_dirty = (map->hdr.flags & MAIL_INDEX_HDR_FLAG_HAVE_DIRTY) != 0;
16532 37bd40e27b67 lib-index: Don't bother tracking if header/records were changed. Timo Sirainen <tss@iki.fi> parents: 16198 diff changeset	987 if (had_dirty)
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	988 map->hdr.flags &= ~MAIL_INDEX_HDR_FLAG_HAVE_DIRTY;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	989
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	990 if (map->hdr_base != map->hdr_copy_buf->data) {
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	991 /* if syncing updates the header, it updates hdr_copy_buf
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	992 and updates hdr_base to hdr_copy_buf. so the buffer must
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	993 initially contain a valid header or we'll break it when
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	994 writing it. */
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	995 buffer_reset(map->hdr_copy_buf);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	996 buffer_append(map->hdr_copy_buf, map->hdr_base,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	997 map->hdr.header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2874 diff changeset	998 map->hdr_base = map->hdr_copy_buf->data;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	999 }
2033 4f6b1118a53d Transaction log contains only UIDs now, no more sequences which just mess up Timo Sirainen <tss@iki.fi> parents: 1972 diff changeset	1000
7563 6de1aed24ce5 Added mail_index_ext_reset_inc() to atomically increase extension's Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	1001 mail_transaction_log_view_get_prev_pos(view->log_view,
6de1aed24ce5 Added mail_index_ext_reset_inc() to atomically increase extension's Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	1002 &prev_seq, &prev_offset);
6de1aed24ce5 Added mail_index_ext_reset_inc() to atomically increase extension's Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	1003
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1004 mail_index_sync_map_init(&sync_map_ctx, view, type);
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1005 if (reset) {
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1006 /* Reset the entire index. Leave only indexid and
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1007 log_file_seq. */
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1008 mail_transaction_log_view_get_prev_pos(view->log_view,
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1009 &prev_seq, &prev_offset);
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1010 map = mail_index_map_alloc(index);
21223 64d9bb10263b lib-index: fsck now adds MAIL_INDEX_HDR_FLAG_FSCKD to header. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21112 diff changeset	1011 if ((index->map->hdr.flags & MAIL_INDEX_HDR_FLAG_FSCKD) != 0)
64d9bb10263b lib-index: fsck now adds MAIL_INDEX_HDR_FLAG_FSCKD to header. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 21112 diff changeset	1012 map->hdr.flags \|= MAIL_INDEX_HDR_FLAG_FSCKD;
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1013 map->hdr.log_file_seq = prev_seq;
6927 9a11ee1c7912 Previous tail_offset change broke when rotating transaction logs. Timo Sirainen <tss@iki.fi> parents: 6924 diff changeset	1014 map->hdr.log_file_tail_offset = 0;
5849 a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1015 mail_index_sync_replace_map(&sync_map_ctx, map);
a9df50952600 Added support for resetting index. Timo Sirainen <tss@iki.fi> parents: 5847 diff changeset	1016 }
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1017 map = NULL;
2935 0b72e95cd3ae Don't try to fsck when we're modifying index. It could have happened with Timo Sirainen <tss@iki.fi> parents: 2932 diff changeset	1018
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1019 /* FIXME: when transaction sync lock is removed, we'll need to handle
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1020 the case when a transaction is committed while mailbox is being
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1021 synced ([synced transactions][new transaction][ext transaction]).
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1022 this means int_offset contains [synced] and ext_offset contains
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1023 all */
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2547 diff changeset	1024 while ((ret = mail_transaction_log_view_next(view->log_view, &thdr,
5728 7aab5d99fb45 Removed sync_mask parameter from mail_transaction_log_view_set(). The user Timo Sirainen <tss@iki.fi> parents: 5725 diff changeset	1025 &tdata)) > 0) {
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1026 mail_transaction_log_view_get_prev_pos(view->log_view,
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1027 &prev_seq, &prev_offset);
2819 bea9c88c8763 Use separate sync offsets for internal/external transactions. Pending external Timo Sirainen <tss@iki.fi> parents: 2762 diff changeset	1028
5749 420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1029 if (LOG_IS_BEFORE(prev_seq, prev_offset,
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1030 view->map->hdr.log_file_seq,
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1031 view->map->hdr.log_file_head_offset)) {
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1032 /* this has been synced already. we're here only to call
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1033 expunge handlers and extension update handlers. */
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1034 i_assert(type == MAIL_INDEX_SYNC_HANDLER_FILE);
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1035
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1036 if ((thdr->type & MAIL_TRANSACTION_EXTERNAL) != 0)
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1037 continue;
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1038 if ((thdr->type & MAIL_TRANSACTION_EXT_MASK) == 0)
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1039 continue;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	1040 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1041
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1042 /* we'll just skip over broken entries */
13428 ad9b826df5a2 lib-index: Make sure mail_index_sync_record() doesn't waste data stack. Timo Sirainen <tss@iki.fi> parents: 13266 diff changeset	1043 (void)mail_index_sync_record(&sync_map_ctx, thdr, tdata);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1044 }
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1045 map = view->map;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1046
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1047 if (had_dirty)
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1048 mail_index_sync_update_hdr_dirty_flag(map);
7620 4b8c1c164d8f Initial CONDSTORE support. Timo Sirainen <tss@iki.fi> parents: 7420 diff changeset	1049 mail_index_modseq_sync_end(&sync_map_ctx.modseq_ctx);
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2260 diff changeset	1050
5777 a8926e90df2a Update log_file_head_offset when replacing current syncing map. Timo Sirainen <tss@iki.fi> parents: 5774 diff changeset	1051 mail_index_sync_update_log_offset(&sync_map_ctx, view->map, TRUE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1052
5785 bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	1053 #ifdef DEBUG
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	1054 mail_index_map_check(map);
bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	1055 #endif
6326 3d7b93aef6b3 Don't assert-crash if uidvalidity wasn't set in first sync. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	1056 i_assert(map->hdr.indexid == index->indexid \|\| map->hdr.indexid == 0);
5785 bcf58c66a099 DEBUG: Verify header counters/lowwaters more often. Timo Sirainen <tss@iki.fi> parents: 5780 diff changeset	1057
5749 420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1058 /* transaction log tracks internally the current tail offset.
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1059 besides using header updates, it also updates the offset to skip
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1060 over following external transactions to avoid extra unneeded log
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to Timo Sirainen <tss@iki.fi> parents: 5746 diff changeset	1061 reading. */
6924 c46a191a8726 If we didn't read tail offset update from transaction log, we set it to 0 in Timo Sirainen <tss@iki.fi> parents: 6821 diff changeset	1062 i_assert(map->hdr.log_file_seq == index->log->head->hdr.file_seq);
c46a191a8726 If we didn't read tail offset update from transaction log, we set it to 0 in Timo Sirainen <tss@iki.fi> parents: 6821 diff changeset	1063 if (map->hdr.log_file_tail_offset < index->log->head->max_tail_offset) {
c46a191a8726 If we didn't read tail offset update from transaction log, we set it to 0 in Timo Sirainen <tss@iki.fi> parents: 6821 diff changeset	1064 map->hdr.log_file_tail_offset =
c46a191a8726 If we didn't read tail offset update from transaction log, we set it to 0 in Timo Sirainen <tss@iki.fi> parents: 6821 diff changeset	1065 index->log->head->max_tail_offset;
c46a191a8726 If we didn't read tail offset update from transaction log, we set it to 0 in Timo Sirainen <tss@iki.fi> parents: 6821 diff changeset	1066 }
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	1067
5867 1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	1068 buffer_write(map->hdr_copy_buf, 0, &map->hdr, sizeof(map->hdr));
1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	1069 if (!MAIL_INDEX_MAP_IS_IN_MEMORY(map)) {
6165 6418139447fa Created a new struct mail_index_record_map which can be shared by multiple Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	1070 memcpy(map->rec_map->mmap_base, map->hdr_copy_buf->data,
5867 1b70ae186611 mmaping works again properly. Changed to use private mmaps which are Timo Sirainen <tss@iki.fi> parents: 5855 diff changeset	1071 map->hdr_copy_buf->used);
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1072 }
8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1073
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1074 /* restore refcount before closing the view. this is necessary also
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1075 if map got cloned, because view closing would otherwise destroy it */
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1076 map->refcount++;
5828 496b7dad3938 Don't access freed memory Timo Sirainen <tss@iki.fi> parents: 5820 diff changeset	1077 mail_index_sync_map_deinit(&sync_map_ctx);
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1078 mail_index_view_close(&view);
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1079
5773 622ab4de7d66 Keep index->map always usable, never NULL. Timo Sirainen <tss@iki.fi> parents: 5772 diff changeset	1080 i_assert(index->map == map \|\| type == MAIL_INDEX_SYNC_HANDLER_VIEW);
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1081
18627 69630e6048fd lib-index: If header is corrupted after syncing, log the reason why. Timo Sirainen <tss@iki.fi> parents: 18398 diff changeset	1082 if (mail_index_map_check_header(map, &error) <= 0) {
6342 702f13f44c5e Check that header is still valid after syncing. If not, fsck. Timo Sirainen <tss@iki.fi> parents: 6326 diff changeset	1083 mail_index_set_error(index,
18627 69630e6048fd lib-index: If header is corrupted after syncing, log the reason why. Timo Sirainen <tss@iki.fi> parents: 18398 diff changeset	1084 "Synchronization corrupted index header %s: %s",
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why. Timo Sirainen <tss@iki.fi> parents: 18398 diff changeset	1085 index->filepath, error);
6342 702f13f44c5e Check that header is still valid after syncing. If not, fsck. Timo Sirainen <tss@iki.fi> parents: 6326 diff changeset	1086 (void)mail_index_fsck(index);
702f13f44c5e Check that header is still valid after syncing. If not, fsck. Timo Sirainen <tss@iki.fi> parents: 6326 diff changeset	1087 map = index->map;
6821 eebfe0cd2add If errors were found while syncing, fsck the index at the end to make sure Timo Sirainen <tss@iki.fi> parents: 6760 diff changeset	1088 } else if (sync_map_ctx.errors) {
eebfe0cd2add If errors were found while syncing, fsck the index at the end to make sure Timo Sirainen <tss@iki.fi> parents: 6760 diff changeset	1089 /* make sure the index looks valid now */
eebfe0cd2add If errors were found while syncing, fsck the index at the end to make sure Timo Sirainen <tss@iki.fi> parents: 6760 diff changeset	1090 (void)mail_index_fsck(index);
eebfe0cd2add If errors were found while syncing, fsck the index at the end to make sure Timo Sirainen <tss@iki.fi> parents: 6760 diff changeset	1091 map = index->map;
6342 702f13f44c5e Check that header is still valid after syncing. If not, fsck. Timo Sirainen <tss@iki.fi> parents: 6326 diff changeset	1092 }
702f13f44c5e Check that header is still valid after syncing. If not, fsck. Timo Sirainen <tss@iki.fi> parents: 6326 diff changeset	1093
5689 c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1094 *_map = map;
c2362f144f15 Initial commit for major index file code cleanup. Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	1095 return ret < 0 ? -1 : 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1096 }

Mercurial > dovecot > core-2.2

annotate src/lib-index/mail-index-sync-update.c @ 22576:707ae9de3812