dovecot/original-hg/dovecot-1.2: src/lib-index/mail-index-strmap.c annotate

annotate src/lib-index/mail-index-strmap.c @ 9458:adee8cb3ff5d HEAD

Minor memory allocation tweaks.

author	Timo Sirainen <tss@iki.fi>
date	Wed, 28 Oct 2009 14:10:04 -0400
parents	2bbf175bb6d3
children	00cd9aacd03c

rev	line source
8590 b9faf4db2a9f Updated copyright notices to include year 2009. Timo Sirainen <tss@iki.fi> parents: 8283 diff changeset	1 /* Copyright (c) 2008-2009 Dovecot authors, see the included COPYING file */
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	4 #include "array.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "bsearch-insert-pos.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	6 #include "istream.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	7 #include "ostream.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	8 #include "file-lock.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	9 #include "file-dotlock.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	10 #include "crc32.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	11 #include "safe-mkstemp.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	12 #include "str.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	13 #include "mail-index-private.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	14 #include "mail-index-strmap.h"
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	15
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	16 #include <stdio.h>
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	17
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	18 struct mail_index_strmap {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	19 struct mail_index *index;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	20 char *path;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	21 int fd;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	22 struct istream *input;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	23
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	24 struct file_lock *file_lock;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	25 struct dotlock *dotlock;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	26 struct dotlock_settings dotlock_settings;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	27 };
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	28
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	29 struct mail_index_strmap_view {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	30 struct mail_index_strmap *strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	31 struct mail_index_view *view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	32
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	33 ARRAY_TYPE(mail_index_strmap_rec) recs;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	34 ARRAY_DEFINE(recs_crc32, uint32_t);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	35 struct hash2_table *hash;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	36
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	37 mail_index_strmap_key_cmp_t *key_compare;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	38 mail_index_strmap_rec_cmp_t *rec_compare;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	39 mail_index_strmap_remap_t *remap_cb;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	40 void *cb_context;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	41
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	42 uoff_t last_read_block_offset;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	43 uint32_t last_read_uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	44 uint32_t last_added_uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	45 uint32_t total_ref_count;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	46
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	47 uint32_t last_ref_index;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	48 uint32_t next_str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	49 uint32_t lost_expunged_uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	50
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	51 unsigned int desynced:1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	52 };
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	53
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	54 struct mail_index_strmap_read_context {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	55 struct mail_index_strmap_view *view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	56
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	57 struct istream *input;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 uoff_t end_offset;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	59 uint32_t highest_str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	60 uint32_t uid_lookup_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	61 uint32_t lost_expunged_uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	62
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	63 const unsigned char data, end, *str_idx_base;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	64 struct mail_index_strmap_rec rec;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	65 uint32_t next_ref_index;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	66 unsigned int rec_size;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	67
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	68 unsigned int too_large_uids:1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	69 };
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	70
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	71 struct mail_index_strmap_view_sync {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	72 struct mail_index_strmap_view *view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	73 };
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	74
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	75 struct mail_index_strmap_hash_key {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	76 const char *str;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	77 uint32_t crc32;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	78 };
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	79
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	80 /* number of bytes required to store one string idx */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	81 #define STRMAP_FILE_STRIDX_SIZE (sizeof(uint32_t)*2)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	82
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	83 /* renumber the string indexes when highest string idx becomes larger than
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	84 <number of indexes>STRMAP_FILE_MAX_STRIDX_MULTIPLIER /
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	85 #define STRMAP_FILE_MAX_STRIDX_MULTIPLIER 2
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	86
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	87 #define STRIDX_MUST_RENUMBER(highest_idx, n_unique_indexes) \
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	88 (highest_idx > n_unique_indexes * STRMAP_FILE_MAX_STRIDX_MULTIPLIER)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	89
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	90 #define MAIL_INDEX_STRMAP_TIMEOUT_SECS 10
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	91
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	92 const struct dotlock_settings default_dotlock_settings = {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	93 MEMBER(temp_prefix) NULL,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	94 MEMBER(lock_suffix) NULL,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	95
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	96 MEMBER(timeout) MAIL_INDEX_STRMAP_TIMEOUT_SECS,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	97 MEMBER(stale_timeout) 30
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	98 };
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	99
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	100 struct mail_index_strmap *
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	101 mail_index_strmap_init(struct mail_index index, const char suffix)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	102 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	103 struct mail_index_strmap *strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	104
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	105 strmap = i_new(struct mail_index_strmap, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	106 strmap->index = index;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	107 strmap->path = i_strconcat(index->filepath, suffix, NULL);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	108 strmap->fd = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	109
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	110 strmap->dotlock_settings = default_dotlock_settings;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	111 strmap->dotlock_settings.use_excl_lock = index->use_excl_dotlocks;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	112 strmap->dotlock_settings.nfs_flush = index->nfs_flush;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	113 return strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	114 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	115
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	116 static bool
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	117 mail_index_strmap_read_rec_next(struct mail_index_strmap_read_context *ctx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	118 uint32_t *crc32_r);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	119
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	120 static void
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	121 mail_index_strmap_set_syscall_error(struct mail_index_strmap *strmap,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	122 const char *function)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	123 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	124 i_assert(function != NULL);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	125
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	126 if (ENOSPACE(errno)) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	127 strmap->index->nodiskspace = TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	128 return;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	129 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	130
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	131 mail_index_set_error(strmap->index,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	132 "%s failed with strmap index file %s: %m",
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	133 function, strmap->path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	134 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	135
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	136 static void mail_index_strmap_close(struct mail_index_strmap *strmap)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	137 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	138 if (strmap->file_lock != NULL)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	139 file_lock_free(&strmap->file_lock);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	140 else if (strmap->dotlock != NULL)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	141 file_dotlock_delete(&strmap->dotlock);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	142
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	143 if (strmap->fd != -1) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	144 if (close(strmap->fd) < 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	145 mail_index_strmap_set_syscall_error(strmap, "close()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	146 strmap->fd = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	147 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	148 if (strmap->input != NULL)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	149 i_stream_unref(&strmap->input);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	150 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	151
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	152 void mail_index_strmap_deinit(struct mail_index_strmap **_strmap)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	153 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	154 struct mail_index_strmap strmap = _strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	155
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	156 *_strmap = NULL;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	157 mail_index_strmap_close(strmap);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	158 i_free(strmap->path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	159 i_free(strmap);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	160 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	161
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	162 static unsigned int mail_index_strmap_hash_key(const void *_key)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	163 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	164 const struct mail_index_strmap_hash_key *key = _key;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	165
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	166 return key->crc32;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	167 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	168
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	169 static bool
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	170 mail_index_strmap_hash_cmp(const void _key, const void _value, void *context)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	171 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	172 const struct mail_index_strmap_hash_key *key = _key;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	173 const struct mail_index_strmap_rec *rec = _value;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	174 struct mail_index_strmap_view *view = context;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	175
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	176 return view->key_compare(key->str, rec, view->cb_context);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	177 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	178
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	179 struct mail_index_strmap_view *
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	180 mail_index_strmap_view_open(struct mail_index_strmap *strmap,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	181 struct mail_index_view *idx_view,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	182 mail_index_strmap_key_cmp_t *key_compare_cb,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	183 mail_index_strmap_rec_cmp_t *rec_compare_cb,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	184 mail_index_strmap_remap_t *remap_cb,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	185 void *context,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	186 const ARRAY_TYPE(mail_index_strmap_rec) **recs_r,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	187 const struct hash2_table **hash_r)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	188 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	189 struct mail_index_strmap_view *view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	190
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	191 view = i_new(struct mail_index_strmap_view, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	192 view->strmap = strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	193 view->view = idx_view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	194 view->key_compare = key_compare_cb;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	195 view->rec_compare = rec_compare_cb;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	196 view->remap_cb = remap_cb;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	197 view->cb_context = context;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	198 view->next_str_idx = 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	199
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	200 i_array_init(&view->recs, 64);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	201 i_array_init(&view->recs_crc32, 64);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	202 view->hash = hash2_create(0, sizeof(struct mail_index_strmap_rec),
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	203 mail_index_strmap_hash_key,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	204 mail_index_strmap_hash_cmp, view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	205 *recs_r = &view->recs;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	206 *hash_r = view->hash;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	207 return view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	208 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	209
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	210 void mail_index_strmap_view_close(struct mail_index_strmap_view **_view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	211 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	212 struct mail_index_strmap_view view = _view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	213
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	214 *_view = NULL;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	215 array_free(&view->recs);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	216 array_free(&view->recs_crc32);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	217 hash2_destroy(&view->hash);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	218 i_free(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	219 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	220
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	221 uint32_t mail_index_strmap_view_get_highest_idx(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	222 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	223 return view->next_str_idx-1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	224 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	225
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	226 static void mail_index_strmap_view_reset(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	227 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	228 view->remap_cb(NULL, 0, 0, view->cb_context);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	229 array_clear(&view->recs);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	230 array_clear(&view->recs_crc32);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	231 hash2_clear(view->hash);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	232
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	233 view->last_added_uid = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	234 view->lost_expunged_uid = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	235 view->desynced = FALSE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	236 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	237
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	238 static void
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	239 mail_index_strmap_view_set_corrupted(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	240 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	241 mail_index_set_error(view->strmap->index,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	242 "Corrupted strmap index file: %s",
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	243 view->strmap->path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	244 (void)unlink(view->strmap->path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	245 mail_index_strmap_close(view->strmap);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	246 mail_index_strmap_view_reset(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	247 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	248
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	249 static int mail_index_strmap_open(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	250 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	251 struct mail_index_strmap *strmap = view->strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	252 const struct mail_index_header *idx_hdr;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	253 struct mail_index_strmap_header hdr;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	254 const unsigned char *data;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	255 size_t size;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	256 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	257
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	258 i_assert(strmap->fd == -1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	259
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	260 strmap->fd = open(strmap->path, O_RDWR);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	261 if (strmap->fd == -1) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	262 if (errno == ENOENT)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	263 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	264 mail_index_strmap_set_syscall_error(strmap, "open()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	265 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	266 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	267 strmap->input = i_stream_create_fd(strmap->fd, (size_t)-1, FALSE);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	268 ret = i_stream_read_data(strmap->input, &data, &size, sizeof(hdr)-1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	269 if (ret <= 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	270 if (ret < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	271 mail_index_strmap_set_syscall_error(strmap, "read()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	272 mail_index_strmap_close(strmap);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	273 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	274 i_assert(ret == 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	275 mail_index_strmap_view_set_corrupted(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	276 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	277 return ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	278 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	279 memcpy(&hdr, data, sizeof(hdr));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	280
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	281 idx_hdr = mail_index_get_header(view->view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	282 if (hdr.version != MAIL_INDEX_STRMAP_VERSION \|\|
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	283 hdr.uid_validity != idx_hdr->uid_validity) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	284 /* need to rebuild. if we already had something in the strmap,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	285 we can keep it. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	286 (void)unlink(strmap->path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	287 mail_index_strmap_close(strmap);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	288 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	289 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	290
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	291 /* we'll read the entire file from the beginning */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	292 view->last_added_uid = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	293 view->last_read_uid = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	294 view->total_ref_count = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	295 view->last_read_block_offset = sizeof(struct mail_index_strmap_header);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	296 view->next_str_idx = 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	297
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	298 mail_index_strmap_view_reset(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	299 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	300 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	301
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	302 static bool mail_index_strmap_need_reopen(struct mail_index_strmap *strmap)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	303 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	304 struct stat st1, st2;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	305
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	306 /* FIXME: nfs flush */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	307 if (fstat(strmap->fd, &st1) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	308 if (errno != ESTALE)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	309 mail_index_strmap_set_syscall_error(strmap, "fstat()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	310 return TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	311 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	312 if (stat(strmap->path, &st2) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	313 mail_index_strmap_set_syscall_error(strmap, "stat()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	314 return TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	315 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	316 return st1.st_ino != st2.st_ino \|\| !CMP_DEV_T(st1.st_dev, st2.st_dev);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	317 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	318
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	319 static int mail_index_strmap_refresh(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	320 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	321 uint32_t seq;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	322
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	323 if (MAIL_INDEX_IS_IN_MEMORY(view->strmap->index))
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	324 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	325
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	326 if (view->strmap->fd != -1) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	327 if (!mail_index_strmap_need_reopen(view->strmap)) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	328 if (view->lost_expunged_uid != 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	329 /* last read failed because view had a message
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	330 that didn't exist in the strmap (because it
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	331 was expunged by another session). if the
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	332 message still isn't expunged in this view,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	333 just continue using the current strmap. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	334 if (mail_index_lookup_seq(view->view,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	335 view->lost_expunged_uid, &seq))
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	336 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	337 } else if (view->desynced) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	338 /* our view isn't synced with the disk, we
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	339 can't read strmap without first resetting
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	340 the view */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	341 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	342 i_stream_sync(view->strmap->input);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	343 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	344 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	345 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	346 mail_index_strmap_close(view->strmap);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	347 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	348
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	349 return mail_index_strmap_open(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	350 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	351
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	352 static int
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	353 mail_index_strmap_read_packed(struct mail_index_strmap_read_context *ctx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	354 uint32_t *num_r)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	355 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	356 const unsigned char *data;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	357 const uint8_t bytes, p, *end;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	358 size_t size;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	359 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	360
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	361 ret = i_stream_read_data(ctx->input, &data, &size, sizeof(*num_r) - 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	362 if (ret <= 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	363 return ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	364
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	365 if (ctx->input->v_offset + size > ctx->end_offset)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	366 size = ctx->end_offset - ctx->input->v_offset;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	367 bytes = p = (const uint8_t *)data;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	368 end = bytes + size;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	369
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	370 if (mail_index_unpack_num(&p, end, num_r) < 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	371 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	372 i_stream_skip(ctx->input, p - bytes);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	373 return 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	374 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	375
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	376 static int
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	377 mail_index_strmap_uid_exists(struct mail_index_strmap_read_context *ctx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	378 uint32_t uid)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	379 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	380 const struct mail_index_record *rec;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	381
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	382 if (ctx->uid_lookup_idx >= ctx->view->view->map->hdr.messages_count) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	383 if (uid >= ctx->view->view->map->hdr.next_uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	384 /* thread index has larger UIDs than what we've seen
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	385 in our view. we'll have to read them again later
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	386 when we know about them */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	387 ctx->too_large_uids = TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	388 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	389 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	390 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	391
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	392 rec = MAIL_INDEX_MAP_IDX(ctx->view->view->map, ctx->uid_lookup_idx);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	393 if (rec->uid == uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	394 ctx->uid_lookup_idx++;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	395 return 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	396 } else if (rec->uid > uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	397 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	398 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	399 /* record that exists in index is missing from strmap.
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	400 see if it's because the strmap is corrupted or because
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	401 our current view is a bit stale and the message has already
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	402 been expunged. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	403 (void)mail_index_refresh(ctx->view->view->index);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	404 if (mail_index_is_expunged(ctx->view->view,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	405 ctx->uid_lookup_idx + 1))
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	406 ctx->lost_expunged_uid = rec->uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	407 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	408 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	409 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	410
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	411 static int
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	412 mail_index_strmap_read_rec_first(struct mail_index_strmap_read_context *ctx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	413 uint32_t *crc32_r)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	414 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	415 size_t size;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	416 uint32_t n, i, count, str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	417 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	418
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	419 /* <uid> <n> <crc32>count <str_idx>count
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	420 where
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	421 n = 0 -> count=1 (only Message-ID:)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	422 n = 1 -> count=2 (Message-ID: + In-Reply-To:)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	423 n = 2+ -> count=n (Message-ID: + References:)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	424 */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	425 if (mail_index_strmap_read_packed(ctx, &n) <= 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	426 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	427 count = n < 2 ? n + 1 : n;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	428 ctx->view->total_ref_count += count;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	429
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	430 ctx->rec_size = count * (sizeof(ctx->rec.str_idx) + sizeof(*crc32_r));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	431 ret = mail_index_strmap_uid_exists(ctx, ctx->rec.uid);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	432 if (ret < 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	433 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	434 if (i_stream_read_data(ctx->view->strmap->input, &ctx->data, &size,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	435 ctx->rec_size - 1) <= 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	436 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	437 ctx->str_idx_base = ctx->data + count * sizeof(uint32_t);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	438
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	439 if (ret == 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	440 /* this message has already been expunged, ignore it.
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	441 update highest string indexes anyway. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	442 for (i = 0; i < count; i++) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	443 memcpy(&str_idx, ctx->str_idx_base, sizeof(str_idx));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	444 if (ctx->highest_str_idx < str_idx)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	445 ctx->highest_str_idx = str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	446 ctx->str_idx_base += sizeof(str_idx);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	447 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	448 i_stream_skip(ctx->view->strmap->input, ctx->rec_size);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	449 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	450 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	451
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	452 /* everything exists. save it. FIXME: these ref_index values
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	453 are thread index specific, perhaps something more generic
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	454 should be used some day */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	455 ctx->end = ctx->data + count * sizeof(*crc32_r);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	456
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	457 ctx->next_ref_index = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	458 if (!mail_index_strmap_read_rec_next(ctx, crc32_r))
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	459 i_unreached();
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	460 ctx->next_ref_index = n == 1 ? 1 : 2;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	461 return 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	462 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	463
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	464 static bool
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	465 mail_index_strmap_read_rec_next(struct mail_index_strmap_read_context *ctx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	466 uint32_t *crc32_r)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	467 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	468 if (ctx->data == ctx->end) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	469 i_stream_skip(ctx->view->strmap->input, ctx->rec_size);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	470 return FALSE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	471 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	472
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	473 /* FIXME: str_idx could be stored as packed relative values
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	474 (first relative to highest_idx, the rest relative to the
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	475 previous str_idx) */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	476
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	477 /* read the record contents */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	478 memcpy(&ctx->rec.str_idx, ctx->str_idx_base, sizeof(ctx->rec.str_idx));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	479 memcpy(crc32_r, ctx->data, sizeof(*crc32_r));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	480
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	481 ctx->rec.ref_index = ctx->next_ref_index++;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	482
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	483 if (ctx->highest_str_idx < ctx->rec.str_idx)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	484 ctx->highest_str_idx = ctx->rec.str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	485
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	486 /* get to the next record */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	487 ctx->data += sizeof(*crc32_r);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	488 ctx->str_idx_base += sizeof(ctx->rec.str_idx);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	489 return TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	490 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	491
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	492 static int
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	493 strmap_read_block_init(struct mail_index_strmap_view *view,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	494 struct mail_index_strmap_read_context *ctx)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	495 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	496 struct mail_index_strmap *strmap = view->strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	497 const unsigned char *data;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	498 size_t size;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	499 uint32_t block_size, seq1, seq2;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	500 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	501
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	502 if (view->last_read_uid + 1 >= view->view->map->hdr.next_uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	503 /* come back later when we know about the new UIDs */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	504 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	505 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	506
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	507 memset(ctx, 0, sizeof(*ctx));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	508 ret = i_stream_read_data(strmap->input, &data, &size,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	509 sizeof(block_size)-1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	510 if (ret <= 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	511 if (strmap->input->stream_errno == 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	512 /* no new data */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	513 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	514 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	515 mail_index_strmap_set_syscall_error(strmap, "read()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	516 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	517 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	518 memcpy(&block_size, data, sizeof(block_size));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	519 block_size = mail_index_offset_to_uint32(block_size) >> 2;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	520 if (block_size == 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	521 /* the rest of the file is either not written, or the previous
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	522 write didn't finish */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	523 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	524 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	525 i_stream_skip(strmap->input, sizeof(block_size));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	526
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	527 ctx->view = view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	528 ctx->input = strmap->input;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	529 ctx->end_offset = strmap->input->v_offset + block_size;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	530 if (ctx->end_offset < strmap->input->v_offset) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	531 /* block size too large */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	532 mail_index_strmap_view_set_corrupted(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	533 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	534 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	535 ctx->rec.uid = view->last_read_uid + 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	536
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	537 /* FIXME: when reading multiple blocks we shouldn't have to calculate
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	538 this every time */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	539 if (!mail_index_lookup_seq_range(view->view, ctx->rec.uid, (uint32_t)-1,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	540 &seq1, &seq2))
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	541 seq1 = mail_index_view_get_messages_count(view->view) + 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	542 ctx->uid_lookup_idx = seq1 - 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	543 return 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	544 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	545
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	546 static int
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	547 strmap_read_block_next(struct mail_index_strmap_read_context *ctx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	548 uint32_t *crc32_r)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	549 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	550 uint32_t uid_diff;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	551 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	552
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	553 if (mail_index_strmap_read_rec_next(ctx, crc32_r))
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	554 return 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	555
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	556 /* get next UID */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	557 do {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	558 if (ctx->input->v_offset == ctx->end_offset) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	559 /* this block is done */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	560 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	561 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	562 if (mail_index_strmap_read_packed(ctx, &uid_diff) < 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	563 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	564
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	565 ctx->rec.uid += uid_diff;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	566 ret = mail_index_strmap_read_rec_first(ctx, crc32_r);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	567 } while (ret == 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	568 return ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	569 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	570
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	571 static int
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	572 strmap_read_block_deinit(struct mail_index_strmap_read_context *ctx, int ret,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	573 bool update_block_offset)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	574 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	575 struct mail_index_strmap_view *view = ctx->view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	576 struct mail_index_strmap *strmap = view->strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	577
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	578 if (ctx->highest_str_idx > view->total_ref_count) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	579 /* if all string indexes are unique, highest_str_index equals
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	580 total_ref_count. otherwise it's always lower. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	581 mail_index_set_error(strmap->index,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	582 "Corrupted strmap index file %s: "
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	583 "String indexes too high "
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	584 "(highest=%u max=%u)",
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	585 strmap->path, ctx->highest_str_idx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	586 view->total_ref_count);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	587 mail_index_strmap_view_set_corrupted(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	588 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	589 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	590 if (ctx->lost_expunged_uid != 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	591 /* our view contained a message that had since been expunged. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	592 i_assert(ret < 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	593 view->lost_expunged_uid = ctx->lost_expunged_uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	594 } else if (ret < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	595 if (strmap->input->stream_errno != 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	596 mail_index_strmap_set_syscall_error(strmap, "read()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	597 else
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	598 mail_index_strmap_view_set_corrupted(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	599 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	600 } else if (update_block_offset && !ctx->too_large_uids) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	601 view->last_read_block_offset = strmap->input->v_offset;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	602 view->last_read_uid = ctx->rec.uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	603 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	604 if (view->next_str_idx <= ctx->highest_str_idx)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	605 view->next_str_idx = ctx->highest_str_idx + 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	606 return ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	607 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	608
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	609 static bool
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	610 strmap_view_sync_handle_conflict(struct mail_index_strmap_read_context *ctx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	611 const struct mail_index_strmap_rec *hash_rec,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	612 struct hash2_iter *iter)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	613 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	614 uint32_t seq;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	615
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	616 /* hopefully it's a message that has since been expunged */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	617 if (!mail_index_lookup_seq(ctx->view->view, hash_rec->uid, &seq)) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	618 /* message is no longer in our view. remove it completely. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	619 hash2_remove_iter(ctx->view->hash, iter);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	620 return TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	621 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	622 if (mail_index_is_expunged(ctx->view->view, seq)) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	623 /* it's quite likely a conflict. we may not be able to verify
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	624 it, so just assume it is. nothing breaks even if we guess
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	625 wrong, the performance just suffers a bit. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	626 return FALSE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	627 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	628
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	629 /* 0 means "doesn't match", which is the only acceptable case */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	630 return ctx->view->rec_compare(&ctx->rec, hash_rec,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	631 ctx->view->cb_context) == 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	632 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	633
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	634 static int
8160 e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	635 strmap_view_sync_block_check_conflicts(struct mail_index_strmap_read_context *ctx,
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	636 uint32_t crc32)
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	637 {
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	638 struct mail_index_strmap_rec *hash_rec;
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	639 struct hash2_iter iter;
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	640
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	641 if (crc32 == 0) {
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	642 /* unique string - there are no conflicts */
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	643 return 0;
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	644 }
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	645
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	646 /* check for conflicting string indexes. they may happen if
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	647
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	648 1) msgid exists only for a message X that has been expunged
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	649 2) another process doesn't see X, but sees msgid for another
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	650 message and writes it using a new string index
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	651 3) if we still see X, we now see the same msgid with two
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	652 string indexes.
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	653
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	654 if we detect such a conflict, we can't continue using the
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	655 strmap index until X has been expunged. */
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	656 memset(&iter, 0, sizeof(iter));
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	657 while ((hash_rec = hash2_iterate(ctx->view->hash,
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	658 crc32, &iter)) != NULL &&
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	659 hash_rec->str_idx != ctx->rec.str_idx) {
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	660 /* CRC32 matches, but string index doesn't */
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	661 if (!strmap_view_sync_handle_conflict(ctx, hash_rec, &iter)) {
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	662 ctx->lost_expunged_uid = hash_rec->uid;
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	663 return -1;
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	664 }
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	665 }
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	666 return 0;
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	667 }
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	668
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	669 static int
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	670 mail_index_strmap_view_sync_block(struct mail_index_strmap_read_context *ctx)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	671 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	672 struct mail_index_strmap_rec *hash_rec;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	673 uint32_t crc32, prev_uid = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	674 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	675
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	676 while ((ret = strmap_read_block_next(ctx, &crc32)) > 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	677 if (ctx->rec.uid <= ctx->view->last_added_uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	678 if (ctx->rec.uid < ctx->view->last_added_uid \|\|
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	679 prev_uid != ctx->rec.uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	680 /* we've already added this */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	681 continue;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	682 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	683 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	684 prev_uid = ctx->rec.uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	685
8160 e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	686 if (strmap_view_sync_block_check_conflicts(ctx, crc32) < 0) {
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	687 ret = -1;
e76d8777d20c strmap: Don't check conflicts for strings known to be unique. Timo Sirainen <tss@iki.fi> parents: 8155 diff changeset	688 break;
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	689 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	690 ctx->view->last_added_uid = ctx->rec.uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	691
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	692 /* add the record to records array */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	693 array_append(&ctx->view->recs, &ctx->rec, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	694 array_append(&ctx->view->recs_crc32, &crc32, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	695
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	696 /* add a separate copy of the record to hash */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	697 hash_rec = hash2_insert_hash(ctx->view->hash, crc32);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	698 memcpy(hash_rec, &ctx->rec, sizeof(*hash_rec));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	699 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	700 return strmap_read_block_deinit(ctx, ret, TRUE);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	701 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	702
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	703 struct mail_index_strmap_view_sync *
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	704 mail_index_strmap_view_sync_init(struct mail_index_strmap_view *view,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	705 uint32_t *last_uid_r)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	706 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	707 struct mail_index_strmap_view_sync *sync;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	708 struct mail_index_strmap_read_context ctx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	709 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	710
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	711 sync = i_new(struct mail_index_strmap_view_sync, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	712 sync->view = view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	713
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	714 if (mail_index_strmap_refresh(view) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	715 /* reading the strmap failed - just ignore and do
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	716 this in-memory based on whatever we knew last */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	717 } else if (view->strmap->input != NULL) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	718 i_stream_seek(view->strmap->input,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	719 view->last_read_block_offset);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	720 while ((ret = strmap_read_block_init(view, &ctx)) > 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	721 if (mail_index_strmap_view_sync_block(&ctx) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	722 ret = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	723 break;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	724 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	725 if (ctx.too_large_uids)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	726 break;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	727 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	728
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	729 if (ret < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	730 /* something failed - we can still use the strmap as far
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	731 as we managed to read it, but our view is now out
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	732 of sync */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	733 view->desynced = TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	734 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	735 i_assert(view->lost_expunged_uid == 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	736 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	737 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	738 *last_uid_r = view->last_added_uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	739 return sync;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	740 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	741
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	742 static inline uint32_t crc32_str_nonzero(const char *str)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	743 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	744 uint32_t value = crc32_str(str);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	745 return value == 0 ? 1 : value;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	746 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	747
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	748 void mail_index_strmap_view_sync_add(struct mail_index_strmap_view_sync *sync,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	749 uint32_t uid, uint32_t ref_index,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	750 const char *key)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	751 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	752 struct mail_index_strmap_view *view = sync->view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	753 struct mail_index_strmap_rec rec, old_rec;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	754 struct mail_index_strmap_hash_key hash_key;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	755 uint32_t str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	756
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	757 i_assert(uid > view->last_added_uid \|\|
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	758 (uid == view->last_added_uid &&
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	759 ref_index > view->last_ref_index));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	760
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	761 hash_key.str = key;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	762 hash_key.crc32 = crc32_str_nonzero(key);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	763
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	764 old_rec = hash2_lookup(view->hash, &hash_key);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	765 if (old_rec != NULL) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	766 /* The string already exists, use the same unique idx */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	767 str_idx = old_rec->str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	768 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	769 /* Newly seen string, assign a new unique idx to it */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	770 str_idx = view->next_str_idx++;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	771 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	772 i_assert(str_idx != 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	773
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	774 rec = hash2_insert(view->hash, &hash_key);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	775 rec->uid = uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	776 rec->ref_index = ref_index;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	777 rec->str_idx = str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	778 array_append(&view->recs, rec, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	779 array_append(&view->recs_crc32, &hash_key.crc32, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	780
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	781 view->last_added_uid = uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	782 view->last_ref_index = ref_index;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	783 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	784
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	785 void mail_index_strmap_view_sync_add_unique(struct mail_index_strmap_view_sync *sync,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	786 uint32_t uid, uint32_t ref_index)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	787 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	788 struct mail_index_strmap_view *view = sync->view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	789 struct mail_index_strmap_rec rec;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	790
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	791 i_assert(uid > view->last_added_uid \|\|
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	792 (uid == view->last_added_uid &&
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	793 ref_index > view->last_ref_index));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	794
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	795 memset(&rec, 0, sizeof(rec));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	796 rec.uid = uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	797 rec.ref_index = ref_index;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	798 rec.str_idx = view->next_str_idx++;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	799 array_append(&view->recs, &rec, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	800 (void)array_append_space(&view->recs_crc32);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	801
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	802 view->last_added_uid = uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	803 view->last_ref_index = ref_index;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	804 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	805
8155 b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	806 static void
b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	807 mail_index_strmap_zero_terminate(struct mail_index_strmap_view *view)
b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	808 {
b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	809 /* zero-terminate the records array */
b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	810 (void)array_append_space(&view->recs);
b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	811 array_delete(&view->recs, array_count(&view->recs)-1, 1);
b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	812 }
b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	813
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	814 static void mail_index_strmap_view_renumber(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	815 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	816 struct mail_index_strmap_read_context ctx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	817 struct mail_index_strmap_rec recs, hash_rec;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	818 uint32_t prev_uid, str_idx, recs_crc32, renumber_map;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	819 unsigned int i, dest, count, count2;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	820 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	821
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	822 memset(&ctx, 0, sizeof(ctx));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	823 ctx.view = view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	824
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	825 /* create a map of old -> new index and remove records of
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	826 expunged messages */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	827 renumber_map = i_new(uint32_t, view->next_str_idx);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	828 str_idx = 0; prev_uid = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	829 recs = array_get_modifiable(&view->recs, &count);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	830 recs_crc32 = array_get_modifiable(&view->recs_crc32, &count2);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	831 i_assert(count == count2);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	832
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	833 for (i = dest = 0; i < count; ) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	834 if (prev_uid != recs[i].uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	835 /* see if this record should be removed */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	836 prev_uid = recs[i].uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	837 ret = mail_index_strmap_uid_exists(&ctx, prev_uid);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	838 i_assert(ret >= 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	839 if (ret == 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	840 /* message expunged */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	841 do {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	842 i++;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	843 } while (i < count && recs[i].uid == prev_uid);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	844 continue;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	845 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	846 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	847
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	848 i_assert(recs[i].str_idx < view->next_str_idx);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	849 if (renumber_map[recs[i].str_idx] == 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	850 renumber_map[recs[i].str_idx] = ++str_idx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	851 if (i != dest) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	852 recs[dest] = recs[i];
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	853 recs_crc32[dest] = recs_crc32[i];
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	854 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	855 i++; dest++;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	856 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	857 i_assert(renumber_map[0] == 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	858 array_delete(&view->recs, dest, i-dest);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	859 array_delete(&view->recs_crc32, dest, i-dest);
8155 b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	860 mail_index_strmap_zero_terminate(view);
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	861
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	862 /* notify caller of the renumbering */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	863 i_assert(str_idx <= view->next_str_idx);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	864 view->remap_cb(renumber_map, view->next_str_idx, str_idx + 1,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	865 view->cb_context);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	866
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	867 /* renumber the indexes in-place and recreate the hash */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	868 recs = array_get_modifiable(&view->recs, &count);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	869 hash2_clear(view->hash);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	870 for (i = 0; i < count; i++) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	871 recs[i].str_idx = renumber_map[recs[i].str_idx];
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	872 hash_rec = hash2_insert_hash(view->hash, recs_crc32[i]);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	873 memcpy(hash_rec, &recs[i], sizeof(*hash_rec));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	874 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	875
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	876 /* update the new next_str_idx only after remapping */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	877 view->next_str_idx = str_idx + 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	878 i_free(renumber_map);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	879 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	880
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	881 static int mail_index_strmap_write_block(struct mail_index_strmap_view *view,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	882 struct ostream *output,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	883 unsigned int i, uint32_t base_uid)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	884 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	885 const struct mail_index_strmap_rec *recs;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	886 const uint32_t *crc32;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	887 unsigned int j, n, count, count2, uid_rec_count;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	888 uint32_t block_size;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	889 uint8_t p, packed[MAIL_INDEX_PACK_MAX_SIZE2];
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	890 uoff_t block_offset, end_offset;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	891
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	892 /* skip over the block size for now, we don't know it yet */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	893 block_offset = output->offset;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	894 block_size = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	895 o_stream_send(output, &block_size, sizeof(block_size));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	896
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	897 /* write records */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	898 recs = array_get(&view->recs, &count);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	899 crc32 = array_get(&view->recs_crc32, &count2);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	900 i_assert(count == count2);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	901 while (i < count) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	902 /* @UNSAFE: <uid diff> */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	903 p = packed;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	904 mail_index_pack_num(&p, recs[i].uid - base_uid);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	905 base_uid = recs[i].uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	906
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	907 /* find how many records belong to this UID */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	908 uid_rec_count = 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	909 for (j = i + 1; j < count; j++) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	910 if (recs[j].uid != base_uid)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	911 break;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	912 uid_rec_count++;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	913 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	914 view->total_ref_count += uid_rec_count;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	915
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	916 /* <n> <crc32>count <str_idx>count -
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	917 FIXME: thread index specific code */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	918 i_assert(recs[i].ref_index == 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	919 if (uid_rec_count == 1) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	920 /* Only Message-ID: header */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	921 n = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	922 } else if (recs[i+1].ref_index == 1) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	923 /* In-Reply-To: header */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	924 n = 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	925 i_assert(uid_rec_count == 2);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	926 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	927 /* References: header */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	928 n = uid_rec_count;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	929 i_assert(recs[i+1].ref_index == 2);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	930 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	931
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	932 mail_index_pack_num(&p, n);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	933 o_stream_send(output, packed, p-packed);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	934 for (j = 0; j < uid_rec_count; j++)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	935 o_stream_send(output, &crc32[i+j], sizeof(crc32[i+j]));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	936 for (j = 0; j < uid_rec_count; j++) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	937 i_assert(j < 2 \|\| recs[i+j].ref_index == j+1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	938 o_stream_send(output, &recs[i+j].str_idx,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	939 sizeof(recs[i+j].str_idx));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	940 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	941 i += uid_rec_count;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	942 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	943
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	944 /* we know the block size now - write it */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	945 block_size = output->offset - (block_offset + sizeof(block_size));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	946 block_size = mail_index_uint32_to_offset(block_size << 2);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	947 i_assert(block_size != 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	948
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	949 end_offset = output->offset;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	950 o_stream_seek(output, block_offset);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	951 o_stream_send(output, &block_size, sizeof(block_size));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	952 o_stream_seek(output, end_offset);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	953
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	954 if (output->last_failed_errno != 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	955 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	956 else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	957 i_assert(view->last_added_uid == recs[count-1].uid);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	958 view->last_read_uid = recs[count-1].uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	959 view->last_read_block_offset = output->offset;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	960 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	961 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	962 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	963
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	964 static void
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	965 mail_index_strmap_recreate_write(struct mail_index_strmap_view *view,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	966 struct ostream *output)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	967 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	968 const struct mail_index_header *idx_hdr;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	969 struct mail_index_strmap_header hdr;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	970
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	971 idx_hdr = mail_index_get_header(view->view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	972
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	973 /* write header */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	974 memset(&hdr, 0, sizeof(hdr));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	975 hdr.version = MAIL_INDEX_STRMAP_VERSION;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	976 hdr.uid_validity = idx_hdr->uid_validity;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	977 o_stream_send(output, &hdr, sizeof(hdr));
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	978
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	979 view->total_ref_count = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	980 (void)mail_index_strmap_write_block(view, output, 0, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	981 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	982
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	983 static int mail_index_strmap_recreate(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	984 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	985 struct mail_index_strmap *strmap = view->strmap;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	986 string_t *str;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	987 struct ostream *output;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	988 const char *temp_path;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	989 int fd, ret = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	990
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	991 if (array_count(&view->recs) == 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	992 /* everything expunged - just unlink the existing index */
8863 b61c15279b65 If unlinking strmap fails because it didn't exist to begin with, don't log an error. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	993 if (unlink(strmap->path) < 0 && errno != ENOENT)
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	994 mail_index_strmap_set_syscall_error(strmap, "unlink()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	995 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	996 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	997
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	998 str = t_str_new(256);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	999 str_append(str, strmap->path);
9168 2bbf175bb6d3 Whenever file's group changing fails, show the group origin in the error message. Timo Sirainen <tss@iki.fi> parents: 8863 diff changeset	1000 fd = safe_mkstemp_hostpid_group(str, view->view->index->mode,
2bbf175bb6d3 Whenever file's group changing fails, show the group origin in the error message. Timo Sirainen <tss@iki.fi> parents: 8863 diff changeset	1001 view->view->index->gid,
2bbf175bb6d3 Whenever file's group changing fails, show the group origin in the error message. Timo Sirainen <tss@iki.fi> parents: 8863 diff changeset	1002 view->view->index->gid_origin);
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1003 temp_path = str_c(str);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1004
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1005 if (fd == -1) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1006 mail_index_set_error(strmap->index,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1007 "safe_mkstemp_hostpid(%s) failed: %m",
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1008 temp_path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1009 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1010 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1011 output = o_stream_create_fd(fd, 0, FALSE);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1012 o_stream_cork(output);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1013 mail_index_strmap_recreate_write(view, output);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1014 if (output->last_failed_errno != 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1015 errno = output->last_failed_errno;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1016 mail_index_set_error(strmap->index,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1017 "write(%s) failed: %m", temp_path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1018 ret = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1019 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1020 o_stream_destroy(&output);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1021 if (close(fd) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1022 mail_index_set_error(strmap->index,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1023 "close(%s) failed: %m", temp_path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1024 ret = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1025 } else if (ret == 0 && rename(temp_path, strmap->path) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1026 mail_index_set_error(strmap->index,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1027 "rename(%s, %s) failed: %m",
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1028 temp_path, strmap->path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1029 ret = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1030 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1031 if (ret < 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1032 (void)unlink(temp_path);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1033 return ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1034 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1035
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1036 static int mail_index_strmap_lock(struct mail_index_strmap *strmap)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1037 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1038 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1039
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1040 i_assert(strmap->fd != -1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1041
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1042 if (strmap->index->lock_method != FILE_LOCK_METHOD_DOTLOCK) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1043 i_assert(strmap->file_lock == NULL);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1044
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1045 ret = file_wait_lock(strmap->fd, strmap->path, F_WRLCK,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1046 strmap->index->lock_method,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1047 MAIL_INDEX_STRMAP_TIMEOUT_SECS,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1048 &strmap->file_lock);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1049 if (ret <= 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1050 mail_index_strmap_set_syscall_error(strmap,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1051 "file_wait_lock()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1052 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1053 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1054 i_assert(strmap->dotlock == NULL);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1055
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1056 ret = file_dotlock_create(&strmap->dotlock_settings,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1057 strmap->path, 0, &strmap->dotlock);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1058 if (ret <= 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1059 mail_index_strmap_set_syscall_error(strmap,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1060 "file_dotlock_create()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1061 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1062 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1063 return ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1064 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1065
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1066 static void mail_index_strmap_unlock(struct mail_index_strmap *strmap)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1067 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1068 if (strmap->file_lock != NULL)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1069 file_unlock(&strmap->file_lock);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1070 else if (strmap->dotlock != NULL)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1071 file_dotlock_delete(&strmap->dotlock);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1072 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1073
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1074 static int strmap_rec_cmp(const void key, const void value)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1075 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1076 const uint32_t *uid = key;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1077 const struct mail_index_strmap_rec *rec = value;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1078
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1079 return *uid < rec->uid ? -1 :
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1080 (*uid > rec->uid ? 1 : 0);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1081 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1082
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1083 static int
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1084 mail_index_strmap_write_append(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1085 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1086 struct mail_index_strmap_read_context ctx;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1087 const struct mail_index_strmap_rec *old_recs;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1088 unsigned int i, old_count;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1089 struct ostream *output;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1090 uint32_t crc32, next_uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1091 bool full_block;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1092 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1093
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1094 /* Check first if another process had written new records to the file.
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1095 If there are any, hopefully they're the same as what we would be
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1096 writing. There are two problematic cases when messages have been
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1097 expunged recently:
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1098
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1099 1) The file contains UIDs that we don't have. This means the string
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1100 indexes won't be compatible anymore, so we'll have to renumber ours
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1101 to match the ones in the strmap file.
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1102
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1103 Currently we don't bother handling 1) case. If indexes don't match
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1104 what we have, we just don't write anything.
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1105
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1106 2) We have UIDs that don't exist in the file. We can't simply skip
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1107 those records, because other records may have pointers to them using
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1108 different string indexes than we have. Even if we renumbered those,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1109 future appends by other processes might cause the same problem (they
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1110 see the string for the first time and assign it a new index, but we
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1111 already have internally given it another index). So the only
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1112 sensible choice is to write nothing and hope that the message goes
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1113 away soon. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1114 old_recs = array_get(&view->recs, &old_count);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1115 next_uid = view->last_read_uid + 1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1116 (void)bsearch_insert_pos(&next_uid, old_recs, old_count,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1117 sizeof(*old_recs), strmap_rec_cmp, &i);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1118 if (i < old_count) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1119 while (i > 0 && old_recs[i-1].uid == old_recs[i].uid)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1120 i--;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1121 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1122
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1123 i_stream_sync(view->strmap->input);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1124 i_stream_seek(view->strmap->input, view->last_read_block_offset);
8283 1e0b1888a876 uninitialized variable fix Timo Sirainen <tss@iki.fi> parents: 8160 diff changeset	1125 full_block = TRUE; ret = 0;
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1126 while (i < old_count &&
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1127 (ret = strmap_read_block_init(view, &ctx)) > 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1128 while ((ret = strmap_read_block_next(&ctx, &crc32)) > 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1129 if (ctx.rec.uid != old_recs[i].uid \|\|
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1130 ctx.rec.str_idx != old_recs[i].str_idx) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1131 /* mismatch */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1132 if (ctx.rec.uid > old_recs[i].uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1133 /* 1) case */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1134 ctx.lost_expunged_uid = ctx.rec.uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1135 } else if (ctx.rec.uid < old_recs[i].uid) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1136 /* 2) case */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1137 ctx.lost_expunged_uid = old_recs[i].uid;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1138 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1139 /* string index mismatch,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1140 shouldn't happen */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1141 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1142 ret = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1143 break;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1144 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1145 if (++i == old_count) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1146 full_block = FALSE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1147 break;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1148 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1149 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1150 if (strmap_read_block_deinit(&ctx, ret, full_block) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1151 ret = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1152 break;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1153 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1154 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1155 if (ret < 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1156 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1157 if (i == old_count) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1158 /* nothing new to write */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1159 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1160 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1161 i_assert(full_block);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1162 i_assert(old_recs[i].uid > view->last_read_uid);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1163
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1164 /* write the new records */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1165 output = o_stream_create_fd(view->strmap->fd, 0, FALSE);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1166 o_stream_seek(output, view->last_read_block_offset);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1167 o_stream_cork(output);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1168 if (mail_index_strmap_write_block(view, output, i,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1169 view->last_read_uid + 1) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1170 errno = output->last_failed_errno;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1171 mail_index_strmap_set_syscall_error(view->strmap, "write()");
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1172 ret = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1173 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1174 o_stream_destroy(&output);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1175 return ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1176 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1177
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1178 static int mail_index_strmap_write(struct mail_index_strmap_view *view)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1179 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1180 int ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1181
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1182 /* FIXME: this renumbering doesn't work well when running for a long
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1183 time since records aren't removed from hash often enough */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1184 if (STRIDX_MUST_RENUMBER(view->next_str_idx - 1,
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1185 hash2_count(view->hash))) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1186 mail_index_strmap_view_renumber(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1187 if (!MAIL_INDEX_IS_IN_MEMORY(view->strmap->index)) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1188 if (mail_index_strmap_recreate(view) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1189 view->desynced = TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1190 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1191 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1192 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1193 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1194 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1195
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1196 if (MAIL_INDEX_IS_IN_MEMORY(view->strmap->index) \|\| view->desynced)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1197 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1198
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1199 if (view->strmap->fd == -1) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1200 /* initial file creation */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1201 if (mail_index_strmap_recreate(view) < 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1202 view->desynced = TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1203 return -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1204 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1205 return 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1206 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1207
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1208 /* append the new records to the strmap file */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1209 if (mail_index_strmap_lock(view->strmap) <= 0) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1210 /* timeout / error */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1211 ret = -1;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1212 } else if (mail_index_strmap_need_reopen(view->strmap)) {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1213 /* the file was already recreated - leave the syncing as it is
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1214 for now and let the next sync re-read the file. */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1215 ret = 0;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1216 } else {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1217 ret = mail_index_strmap_write_append(view);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1218 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1219 mail_index_strmap_unlock(view->strmap);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1220 if (ret < 0)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1221 view->desynced = TRUE;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1222 return ret;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1223 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1224
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1225 void mail_index_strmap_view_sync_commit(struct mail_index_strmap_view_sync **_sync)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1226 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1227 struct mail_index_strmap_view_sync sync = _sync;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1228 struct mail_index_strmap_view *view = sync->view;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1229
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1230 *_sync = NULL;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1231 i_free(sync);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1232
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1233 (void)mail_index_strmap_write(view);
8155 b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	1234 mail_index_strmap_zero_terminate(view);
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1235
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1236 /* zero-terminate the records array */
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1237 (void)array_append_space(&view->recs);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1238 array_delete(&view->recs, array_count(&view->recs)-1, 1);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1239 }
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1240
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1241 void mail_index_strmap_view_sync_rollback(struct mail_index_strmap_view_sync **_sync)
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1242 {
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1243 struct mail_index_strmap_view_sync sync = _sync;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1244
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1245 *_sync = NULL;
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1246
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1247 mail_index_strmap_view_reset(sync->view);
8155 b8feb139cf92 strmap: Make sure the records array is always zero-terminated. Timo Sirainen <tss@iki.fi> parents: 8146 diff changeset	1248 mail_index_strmap_zero_terminate(sync->view);
8146 70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1249 i_free(sync);
70b53e9b232e Rewrote thread indexing code. It's a lot simpler and takes less disk space. Timo Sirainen <tss@iki.fi> parents: diff changeset	1250 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-index-strmap.c @ 9458:adee8cb3ff5d HEAD