dovecot/original-hg/dovecot-1.2: src/lib-index/mail-index.c annotate

annotate src/lib-index/mail-index.c @ 3582:bfa542369fdc HEAD

When creating a new index file, mark its log offsets so that the possibly existing log file is completely synced. This caused marking index to be corrupted to be quite useless since the same problem was again brought back from the .log file.

author	Timo Sirainen <tss@iki.fi>
date	Sun, 18 Sep 2005 18:40:30 +0300
parents	6170e2e5992a
children	575c79a0aabc

rev	line source
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1 /* Copyright (C) 2003-2004 Timo Sirainen */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
3470 346a494c2feb Moved array declaration to array-decl.h and include it in lib.h. So array.h Timo Sirainen <tss@iki.fi> parents: 3445 diff changeset	4 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "buffer.h"
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	6 #include "hash.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	7 #include "mmap-util.h"
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	8 #include "read-full.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	9 #include "write-full.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	10 #include "mail-index-private.h"
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	11 #include "mail-index-sync-private.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	12 #include "mail-transaction-log.h"
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	13 #include "mail-cache.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	14
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	15 #include <stdio.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	16 #include <stddef.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	17 #include <time.h>
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	18 #include <sys/stat.h>
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	19
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	20 static int mail_index_try_open_only(struct mail_index *index);
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	21 static void mail_index_create_in_memory(struct mail_index *index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	22 const struct mail_index_header *hdr);
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	23
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	24 struct mail_index mail_index_alloc(const char dir, const char *prefix)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	25 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	26 struct mail_index *index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	27
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	28 index = i_new(struct mail_index, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	29 index->dir = i_strdup(dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	30 index->prefix = i_strdup(prefix);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	31 index->fd = -1;
2116 fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	32
3294 4600e247da86 Raised initial pool sizes. Timo Sirainen <tss@iki.fi> parents: 3254 diff changeset	33 index->extension_pool = pool_alloconly_create("extension", 512);
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	34 ARRAY_CREATE(&index->extensions, index->extension_pool,
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	35 struct mail_index_ext, 5);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	36
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	37 ARRAY_CREATE(&index->expunge_handlers, default_pool,
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	38 mail_index_expunge_handler_t *, 4);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	39 ARRAY_CREATE(&index->sync_handlers, default_pool,
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	40 struct mail_index_sync_handler, 4);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	41 ARRAY_CREATE(&index->sync_lost_handlers, default_pool,
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	42 mail_index_sync_lost_handler_t *, 4);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	43
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	44 index->mode = 0600;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	45 index->gid = (gid_t)-1;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	46
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	47 index->keywords_ext_id =
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	48 mail_index_ext_register(index, "keywords", 128, 2, 1);
3086 66c44404f9a1 Keyword fixes. Timo Sirainen <tss@iki.fi> parents: 3016 diff changeset	49 index->keywords_pool = pool_alloconly_create("keywords", 512);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	50 ARRAY_CREATE(&index->keywords, default_pool, const char *, 16);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	51 index->keywords_hash =
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	52 hash_create(default_pool, index->keywords_pool, 0,
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	53 strcase_hash, (hash_cmp_callback_t *)strcasecmp);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	54 return index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	55 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	56
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	57 void mail_index_free(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 {
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1965 diff changeset	59 mail_index_close(index);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	60
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	61 hash_destroy(index->keywords_hash);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	62 pool_unref(index->extension_pool);
3086 66c44404f9a1 Keyword fixes. Timo Sirainen <tss@iki.fi> parents: 3016 diff changeset	63 pool_unref(index->keywords_pool);
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1965 diff changeset	64
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	65 array_free(&index->sync_handlers);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	66 array_free(&index->sync_lost_handlers);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	67 array_free(&index->expunge_handlers);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	68 array_free(&index->keywords);
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	69
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	70 i_free(index->error);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	71 i_free(index->dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	72 i_free(index->prefix);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	73 i_free(index);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	74 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	75
2650 9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	76 void mail_index_set_permissions(struct mail_index *index,
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	77 mode_t mode, gid_t gid)
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	78 {
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	79 index->mode = mode & 0666;
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	80 index->gid = gid;
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	81 }
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	82
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	83 uint32_t mail_index_ext_register(struct mail_index index, const char name,
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	84 uint32_t default_hdr_size,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	85 uint16_t default_record_size,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	86 uint16_t default_record_align)
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	87 {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	88 const struct mail_index_ext *extensions;
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	89 struct mail_index_ext ext;
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	90 unsigned int i, ext_count;
2115 f720b3f15333 Modifying extra_records should work now. Timo Sirainen <tss@iki.fi> parents: 2114 diff changeset	91
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	92 extensions = array_get(&index->extensions, &ext_count);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	93
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	94 i_assert(array_count(&index->sync_handlers) == ext_count);
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	95
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	96 /* see if it's already there */
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	97 for (i = 0; i < ext_count; i++) {
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	98 if (strcmp(extensions[i].name, name) == 0)
2116 fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	99 return i;
fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	100 }
fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	101
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	102 memset(&ext, 0, sizeof(ext));
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	103 ext.name = p_strdup(index->extension_pool, name);
2921 a5eda271256f Wrong extension sync_handler might have been called if index extensions Timo Sirainen <tss@iki.fi> parents: 2913 diff changeset	104 ext.index_idx = ext_count;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	105 ext.hdr_size = default_hdr_size;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	106 ext.record_size = default_record_size;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	107 ext.record_align = default_record_align;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	108
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	109 array_append(&index->extensions, &ext, 1);
3445 74475a126e2f Renamed array_modifyable_() functions. I could never remember how they were Timo Sirainen <tss@iki.fi>* parents: 3369 diff changeset	110 (void)array_append_space(&index->sync_handlers);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	111 return ext_count;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	112 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	113
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	114 void mail_index_register_expunge_handler(struct mail_index *index,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	115 uint32_t ext_id,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	116 mail_index_expunge_handler_t *cb)
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	117 {
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	118 mail_index_expunge_handler_t **p;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	119
3445 74475a126e2f Renamed array_modifyable_() functions. I could never remember how they were Timo Sirainen <tss@iki.fi>* parents: 3369 diff changeset	120 p = array_idx_modifyable(&index->expunge_handlers, ext_id);
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	121 i_assert(*p == NULL);
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	122
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	123 *p = cb;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	124 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	125
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	126 void mail_index_unregister_expunge_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	127 uint32_t ext_id)
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	128 {
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	129 mail_index_expunge_handler_t **p;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	130
3445 74475a126e2f Renamed array_modifyable_() functions. I could never remember how they were Timo Sirainen <tss@iki.fi>* parents: 3369 diff changeset	131 p = array_idx_modifyable(&index->expunge_handlers, ext_id);
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	132 i_assert(*p != NULL);
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	133
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	134 *p = NULL;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	135 }
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	136
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	137 void mail_index_register_sync_handler(struct mail_index *index, uint32_t ext_id,
2869 d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2857 diff changeset	138 mail_index_sync_handler_t *cb,
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2857 diff changeset	139 enum mail_index_sync_handler_type type)
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	140 {
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	141 struct mail_index_sync_handler *h;
2869 d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2857 diff changeset	142
3445 74475a126e2f Renamed array_modifyable_() functions. I could never remember how they were Timo Sirainen <tss@iki.fi>* parents: 3369 diff changeset	143 h = array_idx_modifyable(&index->sync_handlers, ext_id);
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	144 i_assert(h->callback == NULL);
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	145
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	146 h->callback = cb;
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	147 h->type = type;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	148 }
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	149
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	150 void mail_index_unregister_sync_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	151 uint32_t ext_id)
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	152 {
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	153 struct mail_index_sync_handler *h;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	154
3445 74475a126e2f Renamed array_modifyable_() functions. I could never remember how they were Timo Sirainen <tss@iki.fi>* parents: 3369 diff changeset	155 h = array_idx_modifyable(&index->sync_handlers, ext_id);
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	156 i_assert(h->callback != NULL);
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	157
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	158 h->callback = NULL;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	159 h->type = 0;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	160 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	161
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	162 void mail_index_register_sync_lost_handler(struct mail_index *index,
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	163 mail_index_sync_lost_handler_t *cb)
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	164 {
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	165 array_append(&index->sync_lost_handlers, &cb, 1);
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	166 }
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	167
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	168 void mail_index_unregister_sync_lost_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	169 mail_index_sync_lost_handler_t *cb)
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	170 {
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	171 mail_index_sync_lost_handler_t const handlers;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	172 unsigned int i, count;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	173
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	174 handlers = array_get(&index->sync_lost_handlers, &count);
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	175 for (i = 0; i < count; i++) {
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	176 if (handlers[i] == cb) {
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	177 array_delete(&index->sync_lost_handlers, i, 1);
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	178 break;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	179 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	180 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	181 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	182
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	183 static void mail_index_map_init_extbufs(struct mail_index_map *map,
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	184 unsigned int initial_count)
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	185 {
3294 4600e247da86 Raised initial pool sizes. Timo Sirainen <tss@iki.fi> parents: 3254 diff changeset	186 #define EXTENSION_NAME_APPROX_LEN 20
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	187 size_t size;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	188
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	189 if (map->extension_pool == NULL) {
3294 4600e247da86 Raised initial pool sizes. Timo Sirainen <tss@iki.fi> parents: 3254 diff changeset	190 size = (sizeof(array_t) + BUFFER_APPROX_SIZE) * 2 +
4600e247da86 Raised initial pool sizes. Timo Sirainen <tss@iki.fi> parents: 3254 diff changeset	191 initial_count * (EXTENSION_NAME_APPROX_LEN +
4600e247da86 Raised initial pool sizes. Timo Sirainen <tss@iki.fi> parents: 3254 diff changeset	192 sizeof(struct mail_index_ext) +
4600e247da86 Raised initial pool sizes. Timo Sirainen <tss@iki.fi> parents: 3254 diff changeset	193 sizeof(uint32_t));
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	194 map->extension_pool =
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	195 pool_alloconly_create("extensions",
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	196 nearest_power(size));
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	197 } else {
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	198 p_clear(map->extension_pool);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	199 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	200
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	201 ARRAY_CREATE(&map->extensions, map->extension_pool,
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	202 struct mail_index_ext, initial_count);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	203 ARRAY_CREATE(&map->ext_id_map, map->extension_pool,
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	204 uint32_t, initial_count);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	205 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	206
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	207 uint32_t mail_index_map_lookup_ext(struct mail_index_map map, const char name)
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	208 {
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	209 const struct mail_index_ext *extensions;
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	210 unsigned int i, size;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	211
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	212 if (!array_is_created(&map->extensions))
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	213 return (uint32_t)-1;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	214
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	215 extensions = array_get(&map->extensions, &size);
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	216 for (i = 0; i < size; i++) {
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	217 if (strcmp(extensions[i].name, name) == 0)
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	218 return i;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	219 }
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	220 return (uint32_t)-1;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	221 }
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	222
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	223 uint32_t
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	224 mail_index_map_register_ext(struct mail_index *index,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	225 struct mail_index_map map, const char name,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	226 uint32_t hdr_offset, uint32_t hdr_size,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	227 uint32_t record_offset, uint32_t record_size,
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	228 uint32_t record_align, uint32_t reset_id)
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	229 {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	230 struct mail_index_ext *ext;
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	231 uint32_t idx, empty_idx = (uint32_t)-1;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	232
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	233 if (!array_is_created(&map->extensions)) {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	234 mail_index_map_init_extbufs(map, 5);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	235 idx = 0;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	236 } else {
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	237 idx = array_count(&map->extensions);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	238 }
3147 6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	239 i_assert(mail_index_map_lookup_ext(map, name) == (uint32_t)-1);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	240
3445 74475a126e2f Renamed array_modifyable_() functions. I could never remember how they were Timo Sirainen <tss@iki.fi>* parents: 3369 diff changeset	241 ext = array_append_space(&map->extensions);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	242 memset(ext, 0, sizeof(*ext));
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	243
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	244 ext->name = p_strdup(map->extension_pool, name);
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	245 ext->hdr_offset = hdr_offset;
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	246 ext->hdr_size = hdr_size;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	247 ext->record_offset = record_offset;
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	248 ext->record_size = record_size;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	249 ext->record_align = record_align;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	250 ext->reset_id = reset_id;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	251
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	252 ext->index_idx = mail_index_ext_register(index, name, hdr_size,
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	253 record_size, record_align);
2921 a5eda271256f Wrong extension sync_handler might have been called if index extensions Timo Sirainen <tss@iki.fi> parents: 2913 diff changeset	254
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	255 while (array_count(&map->ext_id_map) < ext->index_idx)
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	256 array_append(&map->ext_id_map, &empty_idx, 1);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	257 array_idx_set(&map->ext_id_map, ext->index_idx, &idx);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	258 return idx;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	259 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	260
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	261 static int size_check(size_t *size_left, size_t size)
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	262 {
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	263 if (size > *size_left)
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	264 return FALSE;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	265 *size_left -= size;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	266 return TRUE;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	267 }
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	268
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	269 static size_t get_align(size_t name_len)
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	270 {
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	271 size_t size = sizeof(struct mail_index_ext_header) + name_len;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	272 return MAIL_INDEX_HEADER_SIZE_ALIGN(size) - size;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	273 }
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	274
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	275 static int mail_index_read_extensions(struct mail_index *index,
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	276 struct mail_index_map *map)
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	277 {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	278 const struct mail_index_ext_header *ext_hdr;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	279 unsigned int i, old_count;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	280 const char *name;
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	281 uint32_t ext_id, offset, name_offset;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	282 size_t size_left;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	283
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	284 /* extension headers always start from 64bit offsets, so if base header
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	285 doesn't happen to be 64bit aligned we'll skip some bytes */
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	286 offset = MAIL_INDEX_HEADER_SIZE_ALIGN(map->hdr.base_header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	287 if (offset >= map->hdr.header_size && map->extension_pool == NULL) {
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	288 /* nothing to do, skip allocatations and all */
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	289 return 1;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	290 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	291
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	292 old_count = array_count(&index->extensions);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	293 mail_index_map_init_extbufs(map, old_count + 5);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	294
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	295 ext_id = (uint32_t)-1;
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	296 for (i = 0; i < old_count; i++)
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	297 array_append(&map->ext_id_map, &ext_id, 1);
2116 fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	298
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	299 while (offset < map->hdr.header_size) {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	300 ext_hdr = CONST_PTR_OFFSET(map->hdr_base, offset);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	301
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	302 size_left = map->hdr.header_size - offset;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	303 if (!size_check(&size_left, sizeof(*ext_hdr)) \|\|
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	304 !size_check(&size_left, ext_hdr->name_size) \|\|
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	305 !size_check(&size_left, get_align(ext_hdr->name_size)) \|\|
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	306 !size_check(&size_left, ext_hdr->hdr_size)) {
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	307 mail_index_set_error(index, "Corrupted index file %s: "
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	308 "Header extension goes outside header",
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	309 index->filepath);
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	310 return -1;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	311 }
3147 6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	312
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	313 offset += sizeof(*ext_hdr);
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	314 name_offset = offset;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	315 offset += ext_hdr->name_size + get_align(ext_hdr->name_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	316
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	317 t_push();
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	318 name = t_strndup(CONST_PTR_OFFSET(map->hdr_base, name_offset),
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	319 ext_hdr->name_size);
3147 6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	320
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	321 if (mail_index_map_lookup_ext(map, name) != (uint32_t)-1) {
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	322 mail_index_set_error(index, "Corrupted index file %s: "
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	323 "Duplicate header extension %s",
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	324 index->filepath, name);
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	325 t_pop();
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	326 return -1;
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	327 }
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	328
3497 872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	329 if ((ext_hdr->record_offset % ext_hdr->record_align) != 0 \|\|
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	330 (map->hdr.record_size % ext_hdr->record_align) != 0) {
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	331 mail_index_set_error(index, "Corrupted index file %s: "
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	332 "Record field %s alignmentation %u not used",
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	333 index->filepath, name, ext_hdr->record_align);
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	334 t_pop();
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	335 return -1;
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	336 }
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	337
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	338 mail_index_map_register_ext(index, map, name,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	339 offset, ext_hdr->hdr_size,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	340 ext_hdr->record_offset,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	341 ext_hdr->record_size,
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	342 ext_hdr->record_align,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	343 ext_hdr->reset_id);
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	344 t_pop();
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	345
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	346 offset += MAIL_INDEX_HEADER_SIZE_ALIGN(ext_hdr->hdr_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	347 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	348 return 1;
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	349 }
4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	350
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	351 int mail_index_keyword_lookup(struct mail_index *index,
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	352 const char *keyword, int autocreate,
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	353 unsigned int *idx_r)
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	354 {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	355 char *keyword_dup;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	356 void *value;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	357
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	358 if (hash_lookup_full(index->keywords_hash, keyword, NULL, &value)) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	359 *idx_r = POINTER_CAST_TO(value, unsigned int);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	360 return TRUE;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	361 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	362
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	363 if (!autocreate) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	364 *idx_r = (unsigned int)-1;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	365 return FALSE;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	366 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	367
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	368 keyword = keyword_dup = p_strdup(index->keywords_pool, keyword);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	369 *idx_r = array_count(&index->keywords);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	370
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	371 hash_insert(index->keywords_hash, keyword_dup, POINTER_CAST(*idx_r));
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	372 array_append(&index->keywords, &keyword, 1);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	373 return TRUE;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	374 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	375
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	376 int mail_index_map_read_keywords(struct mail_index *index,
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	377 struct mail_index_map *map)
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	378 {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	379 const struct mail_index_ext *ext;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	380 const struct mail_index_keyword_header *kw_hdr;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	381 const struct mail_index_keyword_header_rec *kw_rec;
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	382 const char *name;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	383 unsigned int i, name_len, old_count;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	384 uint32_t ext_id;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	385
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	386 ext_id = mail_index_map_lookup_ext(map, "keywords");
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	387 if (ext_id == (uint32_t)-1) {
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	388 if (array_is_created(&map->keyword_idx_map))
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	389 array_clear(&map->keyword_idx_map);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	390 return 0;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	391 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	392
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	393 ext = array_idx(&map->extensions, ext_id);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	394
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	395 kw_hdr = CONST_PTR_OFFSET(map->hdr_base, ext->hdr_offset);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	396 kw_rec = (const void *)(kw_hdr + 1);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	397 name = (const char *)(kw_rec + kw_hdr->keywords_count);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	398
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	399 old_count = !array_is_created(&map->keyword_idx_map) ? 0 :
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	400 array_count(&map->keyword_idx_map);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	401
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	402 /* Keywords can only be added in mapping. */
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	403 if (kw_hdr->keywords_count == old_count) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	404 /* nothing changed */
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	405 return 0;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	406 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	407
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	408 /* make sure the header is valid */
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	409 if (kw_hdr->keywords_count < old_count) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	410 mail_index_set_error(index, "Corrupted index file %s: "
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	411 "Keywords removed unexpectedly",
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	412 index->filepath);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	413 return -1;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	414 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	415
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	416 if ((size_t)(name - (const char *)kw_hdr) > ext->hdr_size) {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	417 mail_index_set_error(index, "Corrupted index file %s: "
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	418 "keywords_count larger than header size",
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	419 index->filepath);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	420 return -1;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	421 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	422
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	423 name_len = (const char *)kw_hdr + ext->hdr_size - name;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	424 for (i = 0; i < kw_hdr->keywords_count; i++) {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	425 if (kw_rec[i].name_offset > name_len) {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	426 mail_index_set_error(index, "Corrupted index file %s: "
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	427 "name_offset points outside allocated header",
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	428 index->filepath);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	429 return -1;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	430 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	431 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	432 if (name[name_len-1] != '\0') {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	433 mail_index_set_error(index, "Corrupted index file %s: "
3550 6fbdd79b32a5 Error message update Timo Sirainen <tss@iki.fi> parents: 3497 diff changeset	434 "Keyword header doesn't end with NUL",
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	435 index->filepath);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	436 return -1;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	437 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	438
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	439 /* create file -> index mapping */
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	440 if (!array_is_created(&map->keyword_idx_map)) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	441 ARRAY_CREATE(&map->keyword_idx_map, default_pool,
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	442 unsigned int, kw_hdr->keywords_count);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	443 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	444
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	445 #ifdef DEBUG
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	446 for (i = 0; i < array_count(&map->keyword_idx_map); i++) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	447 const char *keyword = name + kw_rec[i].name_offset;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	448 const unsigned int *old_idx;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	449 unsigned int idx;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	450
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	451 old_idx = array_idx(&map->keyword_idx_map, i);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	452 if (!mail_index_keyword_lookup(index, keyword, FALSE, &idx) \|\|
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	453 idx != *old_idx) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	454 mail_index_set_error(index, "Corrupted index file %s: "
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	455 "Keywords changed unexpectedly",
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	456 index->filepath);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	457 return -1;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	458 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	459 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	460 #endif
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	461 i = array_count(&map->keyword_idx_map);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	462 for (; i < kw_hdr->keywords_count; i++) {
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	463 const char *keyword = name + kw_rec[i].name_offset;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	464 unsigned int idx;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	465
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	466 (void)mail_index_keyword_lookup(index, keyword, TRUE, &idx);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	467 array_append(&map->keyword_idx_map, &idx, 1);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	468 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	469 return 0;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	470 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	471
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	472 const array_t mail_index_get_keywords(struct mail_index index)
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	473 {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	474 (void)mail_index_map_read_keywords(index, index->map);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	475 return &index->keywords;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	476 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	477
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	478 static int mail_index_check_header(struct mail_index *index,
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	479 struct mail_index_map *map)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	480 {
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	481 const struct mail_index_header *hdr = &map->hdr;
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	482 unsigned char compat_data[sizeof(hdr->compat_data)];
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	483
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	484 memset(compat_data, 0, sizeof(compat_data));
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	485 #ifndef WORDS_BIGENDIAN
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	486 compat_data[0] = MAIL_INDEX_COMPAT_LITTLE_ENDIAN;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	487 #endif
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	488 compat_data[1] = sizeof(uoff_t);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	489 compat_data[2] = sizeof(time_t);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	490
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	491 if (hdr->major_version != MAIL_INDEX_MAJOR_VERSION) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	492 /* major version change - handle silently(?) */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	493 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	494 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	495 if (memcmp(hdr->compat_data, compat_data, sizeof(compat_data)) != 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	496 /* architecture change - handle silently(?) */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	497 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	498 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	499
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	500 if ((map->hdr.flags & MAIL_INDEX_HDR_FLAG_CORRUPTED) != 0) {
2095 92d8b70829c8 fixes Timo Sirainen <tss@iki.fi> parents: 2073 diff changeset	501 /* we've already complained about it */
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	502 return -1;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	503 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	504
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	505 /* following some extra checks that only take a bit of CPU */
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	506 if (hdr->uid_validity == 0 && hdr->next_uid != 1) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	507 mail_index_set_error(index, "Corrupted index file %s: "
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	508 "uid_validity = 0, next_uid = %u",
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	509 index->filepath, hdr->next_uid);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	510 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	511 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	512
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	513 if (hdr->record_size < sizeof(struct mail_index_record)) {
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	514 mail_index_set_error(index, "Corrupted index file %s: "
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	515 "record_size too small: %u < %"PRIuSIZE_T,
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	516 index->filepath, hdr->record_size,
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	517 sizeof(struct mail_index_record));
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	518 return -1;
4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	519 }
4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	520
3353 286cc0cd888a Check broken flag counter values every time when updating them. Added Timo Sirainen <tss@iki.fi> parents: 3296 diff changeset	521 if ((hdr->flags & MAIL_INDEX_HDR_FLAG_FSCK) != 0)
286cc0cd888a Check broken flag counter values every time when updating them. Added Timo Sirainen <tss@iki.fi> parents: 3296 diff changeset	522 return 0;
286cc0cd888a Check broken flag counter values every time when updating them. Added Timo Sirainen <tss@iki.fi> parents: 3296 diff changeset	523
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	524 if (hdr->next_uid == 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	525 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	526
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2021 diff changeset	527 if (hdr->recent_messages_count > hdr->messages_count \|\|
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2021 diff changeset	528 hdr->seen_messages_count > hdr->messages_count \|\|
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	529 hdr->deleted_messages_count > hdr->messages_count)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	530 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	531 if (hdr->first_recent_uid_lowwater > hdr->next_uid \|\|
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	532 hdr->first_unseen_uid_lowwater > hdr->next_uid \|\|
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	533 hdr->first_deleted_uid_lowwater > hdr->next_uid)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	534 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	535
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	536 return mail_index_read_extensions(index, map);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	537 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	538
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	539 static void mail_index_map_clear(struct mail_index *index,
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	540 struct mail_index_map *map)
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	541 {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	542 if (map->buffer != NULL) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	543 i_assert(map->mmap_base == NULL);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	544 buffer_free(map->buffer);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	545 map->buffer = NULL;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	546 } else if (map->mmap_base != NULL) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	547 i_assert(map->buffer == NULL);
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	548 if (munmap(map->mmap_base, map->mmap_size) < 0)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	549 mail_index_set_syscall_error(index, "munmap()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	550 map->mmap_base = NULL;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	551 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	552
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	553 if (map->refcount > 0) {
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	554 memset(&map->hdr, 0, sizeof(map->hdr));
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	555 map->mmap_size = 0;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	556 map->mmap_used_size = 0;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	557 map->records = NULL;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	558 map->records_count = 0;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	559 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	560 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	561
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	562 void mail_index_unmap(struct mail_index index, struct mail_index_map map)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	563 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	564 if (--map->refcount > 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	565 return;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	566
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	567 i_assert(map->refcount == 0);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	568 mail_index_map_clear(index, map);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	569 if (map->extension_pool != NULL)
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	570 pool_unref(map->extension_pool);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	571 if (array_is_created(&map->keyword_idx_map))
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	572 array_free(&map->keyword_idx_map);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	573 buffer_free(map->hdr_copy_buf);
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	574 i_free(map);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	575 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	576
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	577 static void mail_index_unmap_forced(struct mail_index *index,
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	578 struct mail_index_map *map)
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	579 {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	580 mail_index_map_clear(index, map);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	581 mail_index_unmap(index, map);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	582 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	583
2922 fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	584 static void mail_index_map_copy_hdr(struct mail_index_map *map,
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	585 const struct mail_index_header *hdr)
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	586 {
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	587 if (hdr->base_header_size < sizeof(map->hdr)) {
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	588 /* header smaller than ours, make a copy so our newer headers
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	589 won't have garbage in them */
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	590 memset(&map->hdr, 0, sizeof(map->hdr));
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	591 memcpy(&map->hdr, hdr, hdr->base_header_size);
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	592 } else {
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	593 map->hdr = *hdr;
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	594 }
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	595 }
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	596
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	597 static int mail_index_mmap(struct mail_index index, struct mail_index_map map)
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	598 {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	599 const struct mail_index_header *hdr;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	600 unsigned int records_count;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	601
2193 1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	602 i_assert(!map->write_to_disk);
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	603
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	604 if (map->buffer != NULL) {
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	605 /* we had temporarily used a buffer, eg. for updating index */
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	606 buffer_free(map->buffer);
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	607 map->buffer = NULL;
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	608 }
2043 4548def7ac02 Don't try to re-map dirty memory mapping. Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	609
2761 f96a7b2d1c52 Removed mmap() memory area protections. They weren't always set correctly. Timo Sirainen <tss@iki.fi> parents: 2723 diff changeset	610 map->mmap_base = index->readonly ?
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	611 mmap_ro_file(index->fd, &map->mmap_size) :
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	612 mmap_rw_file(index->fd, &map->mmap_size);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	613 if (map->mmap_base == MAP_FAILED) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	614 map->mmap_base = NULL;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	615 mail_index_set_syscall_error(index, "mmap()");
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	616 return -1;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	617 }
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	618
2822 280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	619 hdr = map->mmap_base;
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	620 if (map->mmap_size >
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	621 offsetof(struct mail_index_header, major_version) &&
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	622 hdr->major_version != MAIL_INDEX_MAJOR_VERSION) {
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	623 /* major version change - handle silently */
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	624 return 0;
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	625 }
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	626
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	627 if (map->mmap_size < MAIL_INDEX_HEADER_MIN_SIZE) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	628 mail_index_set_error(index, "Corrupted index file %s: "
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	629 "File too small (%"PRIuSIZE_T")",
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	630 index->filepath, map->mmap_size);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	631 return 0;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	632 }
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	633
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	634 map->mmap_used_size = hdr->header_size +
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	635 hdr->messages_count * hdr->record_size;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	636
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	637 if (map->mmap_used_size > map->mmap_size) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	638 records_count = (map->mmap_size - hdr->header_size) /
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	639 hdr->record_size;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	640 mail_index_set_error(index, "Corrupted index file %s: "
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	641 "messages_count too large (%u > %u)",
2122 db5807f350ca crashfix on error Timo Sirainen <tss@iki.fi> parents: 2116 diff changeset	642 index->filepath, hdr->messages_count,
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	643 records_count);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	644 return 0;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	645 }
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	646
2922 fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	647 mail_index_map_copy_hdr(map, hdr);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	648
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	649 map->hdr_base = map->mmap_base;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	650 map->records = PTR_OFFSET(map->mmap_base, map->hdr.header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	651 map->records_count = map->hdr.messages_count;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	652 return 1;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	653 }
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	654
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	655 static int mail_index_read_header(struct mail_index *index,
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	656 struct mail_index_header hdr, size_t pos_r)
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	657 {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	658 size_t pos;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	659 int ret;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	660
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	661 memset(hdr, 0, sizeof(*hdr));
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	662
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	663 ret = 1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	664 for (pos = 0; ret > 0 && pos < sizeof(*hdr); ) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	665 ret = pread(index->fd, PTR_OFFSET(hdr, pos),
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	666 sizeof(*hdr) - pos, pos);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	667 if (ret > 0)
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	668 pos += ret;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	669 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	670 *pos_r = pos;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	671 return ret;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	672 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	673
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	674 static int mail_index_read_map(struct mail_index *index,
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	675 struct mail_index_map map, int retry_r)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	676 {
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	677 struct mail_index_header hdr;
1939 4698c121bae3 changes Timo Sirainen <tss@iki.fi> parents: 1935 diff changeset	678 void *data = NULL;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	679 ssize_t ret;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	680 size_t pos, records_size;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	681
2043 4548def7ac02 Don't try to re-map dirty memory mapping. Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	682 i_assert(map->mmap_base == NULL);
4548def7ac02 Don't try to re-map dirty memory mapping. Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	683
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	684 *retry_r = FALSE;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	685 ret = mail_index_read_header(index, &hdr, &pos);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	686
2825 645b8442b519 one more major version change fix Timo Sirainen <tss@iki.fi> parents: 2824 diff changeset	687 if (pos > (ssize_t)offsetof(struct mail_index_header, major_version) &&
2822 280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	688 hdr.major_version != MAIL_INDEX_MAJOR_VERSION) {
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	689 /* major version change - handle silently */
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	690 return 0;
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	691 }
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	692
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	693 if (ret >= 0 && pos >= MAIL_INDEX_HEADER_MIN_SIZE &&
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	694 (ret > 0 \|\| pos >= hdr.base_header_size)) {
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	695 if (hdr.base_header_size < MAIL_INDEX_HEADER_MIN_SIZE \|\|
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	696 hdr.header_size < hdr.base_header_size) {
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	697 mail_index_set_error(index, "Corrupted index file %s: "
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	698 "Corrupted header sizes (base %u, full %u)",
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	699 index->filepath, hdr.base_header_size,
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	700 hdr.header_size);
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	701 return 0;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	702 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	703
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	704 buffer_reset(map->hdr_copy_buf);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	705 buffer_append(map->hdr_copy_buf, &hdr, hdr.base_header_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	706
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	707 /* @UNSAFE */
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	708 data = buffer_append_space_unsafe(map->hdr_copy_buf,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	709 hdr.header_size -
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	710 hdr.base_header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	711 ret = pread_full(index->fd, data,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	712 hdr.header_size - hdr.base_header_size,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	713 hdr.base_header_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	714 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	715
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	716 if (ret > 0) {
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	717 records_size = hdr.messages_count * hdr.record_size;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	718
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	719 if (map->buffer == NULL) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	720 map->buffer = buffer_create_dynamic(default_pool,
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	721 records_size);
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	722 }
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	723
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	724 /* @UNSAFE */
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	725 buffer_set_used_size(map->buffer, 0);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	726 data = buffer_append_space_unsafe(map->buffer, records_size);
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	727
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	728 ret = pread_full(index->fd, data, records_size,
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	729 hdr.header_size);
1939 4698c121bae3 changes Timo Sirainen <tss@iki.fi> parents: 1935 diff changeset	730 }
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	731
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	732 if (ret < 0) {
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	733 if (errno == ESTALE) {
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	734 *retry_r = TRUE;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	735 return 0;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	736 }
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	737 mail_index_set_syscall_error(index, "pread_full()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	738 return -1;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	739 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	740 if (ret == 0) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	741 mail_index_set_error(index,
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	742 "Corrupted index file %s: File too small",
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	743 index->filepath);
2824 431f677bc0b9 More rebuilding fixes with mmap_disable=yes Timo Sirainen <tss@iki.fi> parents: 2823 diff changeset	744 return 0;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	745 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	746
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	747 map->records = data;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	748 map->records_count = hdr.messages_count;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	749
2922 fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	750 mail_index_map_copy_hdr(map, &hdr);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	751 map->hdr_base = map->hdr_copy_buf->data;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	752
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	753 index->sync_log_file_seq = hdr.log_file_seq;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	754 index->sync_log_file_offset = hdr.log_file_int_offset;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	755 return 1;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	756 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	757
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	758 static int mail_index_sync_from_transactions(struct mail_index *index,
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	759 struct mail_index_map **map,
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	760 int sync_to_index)
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	761 {
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	762 const struct mail_index_header map_hdr = &(map)->hdr;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	763 struct mail_index_view *view;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	764 struct mail_transaction_log_view *log_view;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	765 struct mail_index_sync_map_ctx sync_map_ctx;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	766 struct mail_index_header hdr;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	767 const struct mail_transaction_header *thdr;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	768 const void *tdata;
2939 00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	769 uint32_t prev_seq, max_seq;
00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	770 uoff_t prev_offset, max_offset;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	771 size_t pos;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	772 int ret, skipped;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	773
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	774 if (sync_to_index) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	775 /* read the real log position where we are supposed to be
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	776 synced */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	777 ret = mail_index_read_header(index, &hdr, &pos);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	778 if (ret < 0 && errno != ESTALE) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	779 mail_index_set_syscall_error(index, "pread()");
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	780 return -1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	781 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	782 if (pos < MAIL_INDEX_HEADER_MIN_SIZE)
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	783 return 0;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	784
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	785 if (map_hdr->log_file_seq == hdr.log_file_seq &&
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	786 map_hdr->log_file_int_offset == hdr.log_file_int_offset) {
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	787 /* nothing to do */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	788 return 1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	789 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	790
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	791 if (map_hdr->log_file_seq > hdr.log_file_seq \|\|
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	792 (map_hdr->log_file_seq == hdr.log_file_seq &&
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	793 map_hdr->log_file_int_offset > hdr.log_file_int_offset)) {
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	794 /* we went too far, have to re-read the file */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	795 return 0;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	796 }
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	797 if (map_hdr->log_file_ext_offset !=
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	798 map_hdr->log_file_int_offset \|\|
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	799 hdr.log_file_ext_offset != hdr.log_file_int_offset) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	800 /* too much trouble to get this right. */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	801 return 0;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	802 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	803 max_seq = hdr.log_file_seq;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	804 max_offset = hdr.log_file_int_offset;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	805 } else {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	806 /* sync everything there is */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	807 max_seq = (uint32_t)-1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	808 max_offset = (uoff_t)-1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	809 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	810
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	811 log_view = mail_transaction_log_view_open(index->log);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	812 if (mail_transaction_log_view_set(log_view,
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	813 map_hdr->log_file_seq,
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	814 map_hdr->log_file_int_offset,
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	815 max_seq, max_offset,
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	816 MAIL_TRANSACTION_TYPE_MASK) < 0) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	817 mail_transaction_log_view_close(log_view);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	818 return 0;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	819 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	820
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	821 index->map = *map;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	822
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	823 view = mail_index_view_open(index);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	824 mail_index_sync_map_init(&sync_map_ctx, view,
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	825 MAIL_INDEX_SYNC_HANDLER_VIEW);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	826
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	827 while ((ret = mail_transaction_log_view_next(log_view, &thdr, &tdata,
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	828 &skipped)) > 0) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	829 if (mail_index_sync_record(&sync_map_ctx, thdr, tdata) < 0) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	830 ret = -1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	831 break;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	832 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	833 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	834
2939 00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	835 mail_transaction_log_view_get_prev_pos(log_view, &prev_seq,
00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	836 &prev_offset);
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3147 diff changeset	837 i_assert(prev_seq <= max_seq &&
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3147 diff changeset	838 (prev_seq != max_seq \|\| prev_offset <= max_offset));
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3147 diff changeset	839
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3147 diff changeset	840 index->map->hdr.log_file_seq = prev_seq;
2939 00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	841 index->map->hdr.log_file_int_offset =
00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	842 index->map->hdr.log_file_ext_offset = prev_offset;
00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	843
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	844 mail_index_sync_map_deinit(&sync_map_ctx);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	845 mail_index_view_close(view);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	846 mail_transaction_log_view_close(log_view);
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	847
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	848 *map = index->map;
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	849 index->map = NULL;
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	850
2896 305476a6f8ed When we actually read index file's header make sure it gets copied to memory Timo Sirainen <tss@iki.fi> parents: 2895 diff changeset	851 if (sync_to_index) {
3369 b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	852 /* make sure we did everything right. note that although the
b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	853 message counts should be equal, the flag counters may not */
2956 cd257e96a370 Added assert Timo Sirainen <tss@iki.fi> parents: 2940 diff changeset	854 i_assert(hdr.messages_count == (*map)->hdr.messages_count);
3369 b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	855 i_assert(hdr.log_file_seq == (*map)->hdr.log_file_seq);
b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	856 i_assert(hdr.log_file_int_offset == (*map)->hdr.log_file_int_offset);
b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	857 i_assert(hdr.log_file_ext_offset == (*map)->hdr.log_file_ext_offset);
2896 305476a6f8ed When we actually read index file's header make sure it gets copied to memory Timo Sirainen <tss@iki.fi> parents: 2895 diff changeset	858 }
305476a6f8ed When we actually read index file's header make sure it gets copied to memory Timo Sirainen <tss@iki.fi> parents: 2895 diff changeset	859
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	860 return ret < 0 ? -1 : 1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	861 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	862
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	863 static int mail_index_read_map_with_retry(struct mail_index *index,
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	864 struct mail_index_map **map,
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	865 int sync_to_index)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	866 {
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	867 mail_index_sync_lost_handler_t const handlers;
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	868 unsigned int i, count;
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	869 int ret, retry;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	870
2938 bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	871 if (index->log_locked) {
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	872 /* we're most likely syncing the index and we really don't
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	873 want to read more than what was synced last time. */
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	874 sync_to_index = TRUE;
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	875 }
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	876
3177 95d2d8d5958b Some error handling path crashfixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	877 if ((*map)->hdr.indexid != 0 && index->log != NULL) {
95d2d8d5958b Some error handling path crashfixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	878 /* we're not creating the index, or opening transaction log.
95d2d8d5958b Some error handling path crashfixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	879 sync this as a view from transaction log. */
2935 0b72e95cd3ae Don't try to fsck when we're modifying index. It could have happened with Timo Sirainen <tss@iki.fi> parents: 2934 diff changeset	880 index->sync_update = TRUE;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	881 ret = mail_index_sync_from_transactions(index, map,
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	882 sync_to_index);
2935 0b72e95cd3ae Don't try to fsck when we're modifying index. It could have happened with Timo Sirainen <tss@iki.fi> parents: 2934 diff changeset	883 index->sync_update = FALSE;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	884 if (ret != 0)
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	885 return ret;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	886
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	887 /* transaction log lost/broken, fallback to re-reading it */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	888 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	889
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	890 /* notify all "sync lost" handlers */
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	891 handlers = array_get(&index->sync_lost_handlers, &count);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	892 for (i = 0; i < count; i++)
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	893 (*handlers[i])(index);
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	894
1965 41f6a04154a3 INDEX_KEYWORDS_BYTE_COUNT was counted wrong so index files were larger than Timo Sirainen <tss@iki.fi> parents: 1946 diff changeset	895 for (i = 0; i < MAIL_INDEX_ESTALE_RETRY_COUNT; i++) {
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	896 ret = mail_index_read_map(index, *map, &retry);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	897 if (ret != 0 \|\| !retry)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	898 return ret;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	899
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	900 /* ESTALE - reopen index file */
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	901 if (close(index->fd) < 0)
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	902 mail_index_set_syscall_error(index, "close()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	903 index->fd = -1;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	904
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	905 ret = mail_index_try_open_only(index);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	906 if (ret <= 0) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	907 if (ret == 0) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	908 /* the file was lost */
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	909 errno = ENOENT;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	910 mail_index_set_syscall_error(index, "open()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	911 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	912 return -1;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	913 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	914 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	915
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	916 /* Too many ESTALE retries */
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	917 mail_index_set_syscall_error(index, "read_map()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	918 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	919 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	920
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	921 static int mail_index_map_try_existing(struct mail_index_map *map)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	922 {
1946 dbec6c9dbffa syncing fix Timo Sirainen <tss@iki.fi> parents: 1945 diff changeset	923 const struct mail_index_header *hdr;
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	924 size_t used_size;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	925
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	926 if (MAIL_INDEX_MAP_IS_IN_MEMORY(map))
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	927 return 0;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	928
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	929 hdr = map->mmap_base;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	930
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	931 /* always check corrupted-flag to avoid errors later */
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	932 if ((hdr->flags & MAIL_INDEX_HDR_FLAG_CORRUPTED) != 0)
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	933 return -1;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	934
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	935 used_size = hdr->header_size + hdr->messages_count * hdr->record_size;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	936 if (map->mmap_size >= used_size && map->hdr_base == hdr) {
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	937 map->records_count = hdr->messages_count;
2922 fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	938 mail_index_map_copy_hdr(map, hdr);
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	939 return 1;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	940 }
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	941 return 0;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	942 }
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	943
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	944 int mail_index_map(struct mail_index *index, int force)
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	945 {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	946 struct mail_index_map *map;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	947 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	948
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	949 i_assert(!index->mapping);
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	950 i_assert(index->map == NULL \|\| index->map->refcount > 0);
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	951 i_assert(index->lock_type != F_UNLCK);
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	952
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	953 if (MAIL_INDEX_IS_IN_MEMORY(index)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	954 if (index->map == NULL)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	955 mail_index_create_in_memory(index, NULL);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	956 return 1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	957 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	958
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	959 index->mapping = TRUE;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	960
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	961 if (!force && index->map != NULL) {
2937 43deb226d769 Added more asserts. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	962 i_assert(index->hdr != NULL);
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	963 ret = mail_index_map_try_existing(index->map);
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	964 if (ret != 0) {
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	965 index->mapping = FALSE;
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	966 return ret;
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	967 }
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	968
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	969 if (index->lock_type == F_WRLCK) {
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	970 /* we're syncing, don't break the mapping */
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	971 index->mapping = FALSE;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	972 return 1;
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	973 }
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	974 }
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	975
2542 c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	976 if (index->map != NULL && index->map->refcount > 1) {
c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	977 /* this map is already used by some views and they may have
c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	978 pointers into it. leave them and create a new mapping. */
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	979 if (!index->mmap_disable) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	980 map = NULL;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	981 } else {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	982 /* create a copy of the mapping instead so we don't
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	983 have to re-read it */
2911 f2c2b17d7659 Changed mail_index_map_to_memory() to mail_index_map_clone(). Even if the Timo Sirainen <tss@iki.fi> parents: 2902 diff changeset	984 map = mail_index_map_clone(index->map,
f2c2b17d7659 Changed mail_index_map_to_memory() to mail_index_map_clone(). Even if the Timo Sirainen <tss@iki.fi> parents: 2902 diff changeset	985 index->map->hdr.record_size);
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	986 }
2542 c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	987 index->map->refcount--;
c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	988 index->map = NULL;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	989 } else {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	990 map = index->map;
2542 c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	991 }
c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	992
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	993 if (map == NULL) {
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	994 map = i_new(struct mail_index_map, 1);
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	995 map->refcount = 1;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	996 map->hdr_copy_buf =
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	997 buffer_create_dynamic(default_pool, sizeof(map->hdr));
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	998 } else if (MAIL_INDEX_MAP_IS_IN_MEMORY(map)) {
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	999 i_assert(!map->write_to_disk);
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	1000 } else if (map->mmap_base != NULL) {
2193 1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	1001 i_assert(map->buffer == NULL);
1946 dbec6c9dbffa syncing fix Timo Sirainen <tss@iki.fi> parents: 1945 diff changeset	1002 if (munmap(map->mmap_base, map->mmap_size) < 0)
dbec6c9dbffa syncing fix Timo Sirainen <tss@iki.fi> parents: 1945 diff changeset	1003 mail_index_set_syscall_error(index, "munmap()");
dbec6c9dbffa syncing fix Timo Sirainen <tss@iki.fi> parents: 1945 diff changeset	1004 map->mmap_base = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1005 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1006
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1007 index->hdr = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1008 index->map = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1009
2823 43bab10e060d Rebuild broken index files with mmap_disabled=yes too.. Timo Sirainen <tss@iki.fi> parents: 2822 diff changeset	1010 if (!index->mmap_disable)
43bab10e060d Rebuild broken index files with mmap_disabled=yes too.. Timo Sirainen <tss@iki.fi> parents: 2822 diff changeset	1011 ret = mail_index_mmap(index, map);
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1012 else
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	1013 ret = mail_index_read_map_with_retry(index, &map, force);
2937 43deb226d769 Added more asserts. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1014 i_assert(index->map == NULL);
43deb226d769 Added more asserts. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1015
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1016 if (ret > 0) {
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1017 ret = mail_index_check_header(index, map);
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1018 if (ret < 0)
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1019 ret = 0;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1020 else if (ret == 0) {
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1021 index->fsck = TRUE;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1022 ret = 1;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1023 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1024 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1025
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1026 if (ret <= 0) {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1027 mail_index_unmap_forced(index, map);
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1028 index->mapping = FALSE;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1029 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1030 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1031
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1032 index->hdr = &map->hdr;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1033 index->map = map;
2937 43deb226d769 Added more asserts. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1034 i_assert(map->hdr.messages_count == map->records_count);
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1035 index->mapping = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1036 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1037 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1038
3296 c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1039 int mail_index_get_latest_header(struct mail_index *index,
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1040 struct mail_index_header *hdr_r)
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1041 {
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1042 size_t pos;
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1043 int ret;
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1044
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1045 if (!index->mmap_disable) {
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1046 ret = mail_index_map(index, FALSE);
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1047 if (ret > 0)
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1048 hdr_r = index->hdr;
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1049 else
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1050 memset(hdr_r, 0, sizeof(*hdr_r));
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1051 } else {
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1052 ret = mail_index_read_header(index, hdr_r, &pos);
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1053 }
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1054 return ret;
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1055 }
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1056
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	1057 struct mail_index_map *
2911 f2c2b17d7659 Changed mail_index_map_to_memory() to mail_index_map_clone(). Even if the Timo Sirainen <tss@iki.fi> parents: 2902 diff changeset	1058 mail_index_map_clone(struct mail_index_map *map, uint32_t new_record_size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1059 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1060 struct mail_index_map *mem_map;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1061 struct mail_index_header *hdr;
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1062 struct mail_index_ext *extensions;
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1063 void src, dest;
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1064 size_t size, copy_size;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1065 unsigned int i, count;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1066
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1067 size = map->records_count * new_record_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1068
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1069 mem_map = i_new(struct mail_index_map, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1070 mem_map->refcount = 1;
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1071 mem_map->buffer = buffer_create_dynamic(default_pool, size);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1072 if (map->hdr.record_size == new_record_size)
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1073 buffer_append(mem_map->buffer, map->records, size);
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1074 else {
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1075 copy_size = I_MIN(map->hdr.record_size, new_record_size);
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1076 src = map->records;
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1077 for (i = 0; i < map->records_count; i++) {
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1078 dest = buffer_append_space_unsafe(mem_map->buffer,
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1079 new_record_size);
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1080 memcpy(dest, src, copy_size);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1081 src = PTR_OFFSET(src, map->hdr.record_size);
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1082 }
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1083 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1084
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1085 mem_map->records = buffer_get_modifyable_data(mem_map->buffer, NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1086 mem_map->records_count = map->records_count;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1087
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1088 mem_map->hdr_copy_buf =
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1089 buffer_create_dynamic(default_pool, map->hdr.header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1090 if (map->hdr.base_header_size < sizeof(*hdr))
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1091 buffer_append_zero(mem_map->hdr_copy_buf, sizeof(*hdr));
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1092 buffer_write(mem_map->hdr_copy_buf, 0,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1093 &map->hdr, map->hdr.base_header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1094 buffer_append(mem_map->hdr_copy_buf,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1095 CONST_PTR_OFFSET(map->hdr_base,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1096 map->hdr.base_header_size),
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1097 map->hdr.header_size - map->hdr.base_header_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1098
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1099 hdr = buffer_get_modifyable_data(mem_map->hdr_copy_buf, NULL);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1100 if (hdr->base_header_size < sizeof(*hdr))
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1101 hdr->base_header_size = sizeof(*hdr);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1102 hdr->record_size = new_record_size;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1103 mem_map->hdr = *hdr;
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1104 mem_map->hdr_base = hdr;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1105
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1106 /* copy extensions */
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1107 if (array_is_created(&map->ext_id_map)) {
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1108 count = array_count(&map->ext_id_map);
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	1109 mail_index_map_init_extbufs(mem_map, count + 2);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1110
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1111 array_append_array(&mem_map->extensions, &map->extensions);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1112 array_append_array(&mem_map->ext_id_map, &map->ext_id_map);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1113
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1114 /* fix the name pointers to use our own pool */
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1115 extensions = array_get_modifyable(&mem_map->extensions, &count);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1116 for (i = 0; i < count; i++) {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1117 extensions[i].name = p_strdup(mem_map->extension_pool,
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1118 extensions[i].name);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1119 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1120 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1121
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1122 return mem_map;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1123 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1124
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1125 int mail_index_map_get_ext_idx(struct mail_index_map *map,
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1126 uint32_t ext_id, uint32_t *idx_r)
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1127 {
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1128 const uint32_t *id;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1129
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1130 if (!array_is_created(&map->ext_id_map) \|\|
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1131 ext_id >= array_count(&map->ext_id_map))
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1132 return 0;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1133
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1134 id = array_idx(&map->ext_id_map, ext_id);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1135 idx_r = id;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1136 return *idx_r != (uint32_t)-1;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1137 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1138
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1139 static int mail_index_try_open_only(struct mail_index *index)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1140 {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1141 int i;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1142
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1143 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1144
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1145 for (i = 0; i < 3; i++) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1146 index->fd = open(index->filepath, O_RDWR);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1147 if (index->fd == -1 && errno == EACCES) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1148 index->fd = open(index->filepath, O_RDONLY);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1149 index->readonly = TRUE;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1150 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1151 if (index->fd != -1 \|\| errno != ESTALE)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1152 break;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1153
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1154 /* May happen with some OSes with NFS. Try again, although
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1155 there's still a race condition with another computer
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1156 creating the index file again. However, we can't try forever
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1157 as ESTALE happens also if index directory has been deleted
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1158 from server.. */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1159 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1160 if (index->fd == -1) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1161 if (errno != ENOENT)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1162 return mail_index_set_syscall_error(index, "open()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1163
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1164 /* have to create it */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1165 return 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1166 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1167 return 1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1168 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1169
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1170 static int
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1171 mail_index_try_open(struct mail_index index, unsigned int lock_id_r)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1172 {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1173 unsigned int lock_id;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1174 int ret;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1175
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1176 if (lock_id_r != NULL)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1177 *lock_id_r = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1178
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1179 if (MAIL_INDEX_IS_IN_MEMORY(index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1180 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1181
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1182 ret = mail_index_try_open_only(index);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1183 if (ret <= 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1184 return ret;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1185
2409 9183098d7fdf Fixed fd leaks Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	1186 if (mail_index_lock_shared(index, FALSE, &lock_id) < 0) {
9183098d7fdf Fixed fd leaks Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	1187 (void)close(index->fd);
9183098d7fdf Fixed fd leaks Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	1188 index->fd = -1;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1189 return -1;
2409 9183098d7fdf Fixed fd leaks Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	1190 }
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1191 ret = mail_index_map(index, FALSE);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1192 if (ret == 0) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1193 /* it's corrupted - recreate it */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1194 mail_index_unlock(index, lock_id);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1195 if (lock_id_r != NULL)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1196 *lock_id_r = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1197
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1198 (void)close(index->fd);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1199 index->fd = -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1200 } else {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1201 if (lock_id_r != NULL)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1202 *lock_id_r = lock_id;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1203 else
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1204 mail_index_unlock(index, lock_id);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1205 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1206 return ret;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1207 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1208
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1209 int mail_index_write_base_header(struct mail_index *index,
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1210 const struct mail_index_header *hdr)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1211 {
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1212 size_t hdr_size;
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1213
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1214 hdr_size = I_MIN(sizeof(*hdr), hdr->base_header_size);
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1215
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1216 if (!MAIL_INDEX_MAP_IS_IN_MEMORY(index->map)) {
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1217 memcpy(index->map->mmap_base, hdr, hdr_size);
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1218 if (msync(index->map->mmap_base, hdr_size, MS_SYNC) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1219 return mail_index_set_syscall_error(index, "msync()");
3222 e38e04c7c67c mail_index_write_base_header() didn't update the header in memory with Timo Sirainen <tss@iki.fi> parents: 3191 diff changeset	1220 index->map->hdr = *hdr;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1221 } else {
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1222 if (!MAIL_INDEX_IS_IN_MEMORY(index)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1223 if (pwrite_full(index->fd, hdr, hdr_size, 0) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1224 mail_index_set_syscall_error(index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1225 "pwrite_full()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1226 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1227 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1228 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1229
2902 1022caefca74 mail_index_write_base_header() doesn't crash anymore if index mapping is in Timo Sirainen <tss@iki.fi> parents: 2896 diff changeset	1230 index->map->hdr = *hdr;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1231 buffer_write(index->map->hdr_copy_buf, 0, hdr, hdr_size);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1232 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1233
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1234 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1235 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1236
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1237 int mail_index_create_tmp_file(struct mail_index index, const char *path_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1238 {
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1239 mode_t old_mask;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1240 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1241 int fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1242
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1243 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1244
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1245 path = *path_r = t_strconcat(index->filepath, ".tmp", NULL);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1246 old_mask = umask(0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1247 fd = open(path, O_RDWR\|O_CREAT\|O_TRUNC, index->mode);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1248 umask(old_mask);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1249 if (fd == -1)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1250 return mail_index_file_set_syscall_error(index, path, "open()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1251
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1252 if (index->gid != (gid_t)-1 && fchown(fd, (uid_t)-1, index->gid) < 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1253 mail_index_file_set_syscall_error(index, path, "fchown()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1254 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1255 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1256
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1257 return fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1258 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1259
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1260 static int mail_index_create(struct mail_index *index,
2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1261 struct mail_index_header *hdr)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1262 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1263 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1264 uint32_t seq;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1265 uoff_t offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1266 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1267
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1268 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1269 i_assert(index->lock_type == F_UNLCK);
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1270
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1271 /* log file lock protects index creation */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1272 if (mail_transaction_log_sync_lock(index->log, &seq, &offset) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1273 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1274
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1275 ret = mail_index_try_open(index, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1276 if (ret != 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1277 mail_transaction_log_sync_unlock(index->log);
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1278 return ret < 0 ? -1 : 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1279 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1280
3582 bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1281 /* mark the existing log file as synced */
bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1282 hdr->log_file_seq = seq;
bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1283 hdr->log_file_int_offset = offset;
bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1284 hdr->log_file_ext_offset = offset;
bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1285
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1286 /* create it fully in index.tmp first */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1287 index->fd = mail_index_create_tmp_file(index, &path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1288 if (index->fd == -1)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1289 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1290 else if (write_full(index->fd, hdr, sizeof(*hdr)) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1291 mail_index_file_set_syscall_error(index, path, "write_full()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1292 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1293 } else {
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1294 index->lock_type = F_WRLCK;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1295 ret = mail_index_map(index, FALSE);
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1296 index->lock_type = F_UNLCK;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1297 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1298
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1299 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1300 /* it's corrupted even while we just created it,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1301 should never happen unless someone pokes the file directly */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1302 mail_index_set_error(index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1303 "Newly created index file is corrupted: %s", path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1304 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1305 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1306
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1307 if (ret < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1308 if (unlink(path) < 0 && errno != ENOENT) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1309 mail_index_file_set_syscall_error(index, path,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1310 "unlink()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1311 }
2170 fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1312 } else {
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1313 /* make it visible to others */
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1314 if (rename(path, index->filepath) < 0) {
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1315 mail_index_set_error(index, "rename(%s, %s) failed: %m",
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1316 path, index->filepath);
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1317 ret = -1;
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1318 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1319 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1320
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1321 mail_transaction_log_sync_unlock(index->log);
2170 fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1322 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1323 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1324
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1325 static void mail_index_header_init(struct mail_index_header *hdr)
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1326 {
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1327 time_t now = time(NULL);
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1328
3105 f58ff8ec50c4 Added asserts Timo Sirainen <tss@iki.fi> parents: 3086 diff changeset	1329 i_assert((sizeof(*hdr) % sizeof(uint64_t)) == 0);
f58ff8ec50c4 Added asserts Timo Sirainen <tss@iki.fi> parents: 3086 diff changeset	1330
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1331 memset(hdr, 0, sizeof(*hdr));
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1332
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1333 hdr->major_version = MAIL_INDEX_MAJOR_VERSION;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1334 hdr->minor_version = MAIL_INDEX_MINOR_VERSION;
2116 fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	1335 hdr->base_header_size = sizeof(*hdr);
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1336 hdr->header_size = sizeof(*hdr);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1337 hdr->record_size = sizeof(struct mail_index_record);
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1338
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1339 #ifndef WORDS_BIGENDIAN
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1340 hdr->compat_data[0] = MAIL_INDEX_COMPAT_LITTLE_ENDIAN;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1341 #endif
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1342 hdr->compat_data[1] = sizeof(uoff_t);
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1343 hdr->compat_data[2] = sizeof(time_t);
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1344
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1345 hdr->indexid = now;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1346
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1347 hdr->next_uid = 1;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1348 }
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1349
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1350 static void mail_index_create_in_memory(struct mail_index *index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1351 const struct mail_index_header *hdr)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1352 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1353 struct mail_index_header tmp_hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1354 struct mail_index_map tmp_map;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1355
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1356 if (hdr == NULL) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1357 mail_index_header_init(&tmp_hdr);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1358 hdr = &tmp_hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1359 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1360
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1361 memset(&tmp_map, 0, sizeof(tmp_map));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1362 tmp_map.hdr = *hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1363 tmp_map.hdr_base = hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1364
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1365 /* a bit kludgy way to do this, but it initializes everything
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1366 nicely and correctly */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1367 index->map = mail_index_map_clone(&tmp_map, hdr->record_size);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1368 index->hdr = &index->map->hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1369 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1370
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1371 /* returns -1 = error, 0 = won't create, 1 = ok */
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1372 static int mail_index_open_files(struct mail_index *index,
2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1373 enum mail_index_open_flags flags)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1374 {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1375 struct mail_index_header hdr;
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1930 diff changeset	1376 unsigned int lock_id = 0;
3129 1ea614abead3 Don't complain about cache indexid changing when recreating index. Timo Sirainen <tss@iki.fi> parents: 3105 diff changeset	1377 int ret, created = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1378
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1930 diff changeset	1379 ret = mail_index_try_open(index, &lock_id);
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1380 if (ret > 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1381 hdr = *index->hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1382 else if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1383 /* doesn't exist, or corrupted */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1384 if ((flags & MAIL_INDEX_OPEN_FLAG_CREATE) == 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1385 return 0;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1386 mail_index_header_init(&hdr);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1387 index->hdr = &hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1388 } else if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1389 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1390
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1391 index->indexid = hdr.indexid;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1392
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1393 index->log = mail_transaction_log_open_or_create(index);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1394 if (index->log == NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1395 return -1;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1396
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1397 if (index->fd == -1) {
3556 6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	1398 if (index->indexid != hdr.indexid) {
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	1399 /* looks like someone else created the transaction log
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	1400 before we had the chance. use its indexid so we
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	1401 don't try to create conflicting ones. */
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	1402 hdr.indexid = index->indexid;
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	1403 }
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	1404
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1405 if (lock_id != 0) {
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1406 mail_index_unlock(index, lock_id);
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1407 lock_id = 0;
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1408 }
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1409 if (!MAIL_INDEX_IS_IN_MEMORY(index)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1410 if (mail_index_create(index, &hdr) < 0)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1411 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1412 } else {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1413 mail_index_create_in_memory(index, &hdr);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1414 }
3129 1ea614abead3 Don't complain about cache indexid changing when recreating index. Timo Sirainen <tss@iki.fi> parents: 3105 diff changeset	1415 created = TRUE;
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1416 }
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1417
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1418 if (lock_id == 0) {
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1419 if (mail_index_lock_shared(index, FALSE, &lock_id) < 0)
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1420 return -1;
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1421
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1422 }
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1423
3129 1ea614abead3 Don't complain about cache indexid changing when recreating index. Timo Sirainen <tss@iki.fi> parents: 3105 diff changeset	1424 index->cache = created ? mail_cache_create(index) :
1ea614abead3 Don't complain about cache indexid changing when recreating index. Timo Sirainen <tss@iki.fi> parents: 3105 diff changeset	1425 mail_cache_open_or_create(index);
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1426
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1427 mail_index_unlock(index, lock_id);
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1428 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1429 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1430
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2715 diff changeset	1431 int mail_index_open(struct mail_index *index, enum mail_index_open_flags flags,
12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2715 diff changeset	1432 enum mail_index_lock_method lock_method)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1433 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1434 int i = 0, ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1435
2926 c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1436 if (index->opened) {
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1437 if (index->hdr != NULL &&
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1438 (index->hdr->flags & MAIL_INDEX_HDR_FLAG_CORRUPTED) != 0) {
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1439 /* corrupted, reopen files */
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1440 mail_index_close(index);
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1441 } else {
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1442 return 0;
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1443 }
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1444 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1445
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1446 index->filepath = MAIL_INDEX_IS_IN_MEMORY(index) ?
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1447 i_strdup("(in-memory index)") :
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1448 i_strconcat(index->dir, "/", index->prefix, NULL);
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1965 diff changeset	1449
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1450 do {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1451 index->shared_lock_count = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1452 index->excl_lock_count = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1453 index->lock_type = F_UNLCK;
2021 19e638ade25c Locking fixes Timo Sirainen <tss@iki.fi> parents: 1984 diff changeset	1454 index->lock_id = 2;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1455
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2715 diff changeset	1456 index->readonly = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1457 index->nodiskspace = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1458 index->index_lock_timeout = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1459 index->log_locked = FALSE;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1460 index->mmap_disable =
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1461 (flags & MAIL_INDEX_OPEN_FLAG_MMAP_DISABLE) != 0;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1462 index->mmap_no_write =
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1463 (flags & MAIL_INDEX_OPEN_FLAG_MMAP_NO_WRITE) != 0;
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2715 diff changeset	1464 index->lock_method = lock_method;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1465
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1466 /* don't even bother to handle dotlocking without mmap being
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1467 disabled. that combination simply doesn't make any sense */
2940 0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1468 if (lock_method == MAIL_INDEX_LOCK_DOTLOCK &&
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1469 !index->mmap_disable) {
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1470 i_fatal("lock_method=dotlock and mmap_disable=no "
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1471 "combination isn't supported. "
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1472 "You don't _really_ want it anyway.");
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1473 }
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1474
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1475 ret = mail_index_open_files(index, flags);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1476 if (ret <= 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1477 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1478
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1479 index->opened = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1480 if (index->fsck) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1481 index->fsck = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1482 ret = mail_index_fsck(index);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1483 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1484 /* completely broken, reopen */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1485 if (i++ < 3)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1486 continue;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1487 /* too many tries */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1488 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1489 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1490 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1491 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1492 } while (1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1493
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1494 if (ret <= 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1495 mail_index_close(index);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1496
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1497 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1498 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1499
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1500 void mail_index_close(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1501 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1502 if (index->log != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1503 mail_transaction_log_close(index->log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1504 index->log = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1505 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1506
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	1507 if (index->map != NULL) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	1508 mail_index_unmap(index, index->map);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	1509 index->map = NULL;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	1510 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1511
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1512 if (index->cache != NULL) {
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1513 mail_cache_free(index->cache);
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1514 index->cache = NULL;
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1515 }
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1516
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1517 if (index->fd != -1) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1518 if (close(index->fd) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1519 mail_index_set_syscall_error(index, "close()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1520 index->fd = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1521 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1522
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1523 i_free(index->copy_lock_path);
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	1524 index->copy_lock_path = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1525 i_free(index->filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1526 index->filepath = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1527
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1528 index->indexid = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1529 index->opened = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1530 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1531
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1532 int mail_index_reopen(struct mail_index *index, int fd)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1533 {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1534 struct mail_index_map *old_map;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1535 unsigned int old_shared_locks, old_lock_id, lock_id = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1536 int ret, old_fd, old_lock_type;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1537
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1538 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1539
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1540 old_map = index->map;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1541 old_fd = index->fd;
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1542 old_map->refcount++;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1543
3551 65dc45982df9 Removed fcntl() text from some locking error messages, because if Timo Sirainen <tss@iki.fi> parents: 3550 diff changeset	1544 /* new file, new locks. the old fd can keep its locks, they don't
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1545 matter anymore as no-one's going to modify the file. */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1546 old_lock_type = index->lock_type;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1547 old_lock_id = index->lock_id;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1548 old_shared_locks = index->shared_lock_count;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1549
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1550 if (index->lock_type == F_RDLCK)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1551 index->lock_type = F_UNLCK;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1552 index->lock_id += 2;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1553 index->shared_lock_count = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1554
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1555 if (fd != -1) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1556 index->fd = fd;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1557 ret = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1558 } else {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1559 i_assert(index->excl_lock_count == 0);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1560 ret = mail_index_try_open_only(index);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1561 if (ret > 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1562 ret = mail_index_lock_shared(index, FALSE, &lock_id);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1563 else if (ret == 0) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1564 /* index file is lost */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1565 ret = -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1566 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1567 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1568
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1569 if (ret == 0) {
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1570 /* read the new mapping. note that with mmap_disable we want
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1571 to keep the old mapping in index->map so we can update it
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1572 by reading transaction log. */
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1573 if (mail_index_map(index, TRUE) <= 0)
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1574 ret = -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1575 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1576
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1577 if (lock_id != 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1578 mail_index_unlock(index, lock_id);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1579
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1580 if (ret == 0) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1581 mail_index_unmap(index, old_map);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1582 if (close(old_fd) < 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1583 mail_index_set_syscall_error(index, "close()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1584 } else {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1585 if (index->map != NULL)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1586 mail_index_unmap(index, index->map);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1587 if (index->fd != -1) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1588 if (close(index->fd) < 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1589 mail_index_set_syscall_error(index, "close()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1590 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1591
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1592 index->map = old_map;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1593 index->hdr = &index->map->hdr;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1594 index->fd = old_fd;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1595 index->lock_type = old_lock_type;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1596 index->lock_id = old_lock_id;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1597 index->shared_lock_count = old_shared_locks;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1598 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1599 return ret;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1600 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1601
2936 57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1602 int mail_index_reopen_if_needed(struct mail_index *index)
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1603 {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1604 struct stat st1, st2;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1605
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1606 if (MAIL_INDEX_IS_IN_MEMORY(index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1607 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1608
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1609 if (fstat(index->fd, &st1) < 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1610 return mail_index_set_syscall_error(index, "fstat()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1611 if (stat(index->filepath, &st2) < 0) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1612 mail_index_set_syscall_error(index, "stat()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1613 if (errno != ENOENT)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1614 return -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1615
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1616 /* lost it? recreate */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1617 (void)mail_index_mark_corrupted(index);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1618 return -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1619 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1620
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1621 if (st1.st_ino != st2.st_ino \|\|
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1622 !CMP_DEV_T(st1.st_dev, st2.st_dev)) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1623 if (mail_index_reopen(index, -1) < 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1624 return -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1625 return 1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1626 } else {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1627 return 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1628 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1629 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1630
2936 57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1631 int mail_index_refresh(struct mail_index *index)
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1632 {
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1633 unsigned int lock_id;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1634 int ret;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1635
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1636 if (MAIL_INDEX_IS_IN_MEMORY(index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1637 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1638
2936 57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1639 if (index->excl_lock_count > 0) {
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1640 /* we have index exclusively locked, nothing could
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1641 have changed. */
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1642 return 0;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1643 }
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1644
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1645 if (!index->mmap_disable) {
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1646 /* reopening is all we need */
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1647 return mail_index_reopen_if_needed(index);
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1648 }
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1649
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	1650 i_assert(!index->mapping);
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	1651
2936 57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1652 /* mail_index_map() simply reads latest changes from transaction log,
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1653 which makes us fully refreshed. */
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1654 if (mail_index_lock_shared(index, TRUE, &lock_id) < 0)
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1655 return -1;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1656
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1657 ret = mail_index_map(index, FALSE);
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1658 mail_index_unlock(index, lock_id);
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1659 return ret <= 0 ? -1 : 0;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1660 }
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1661
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1662 struct mail_cache mail_index_get_cache(struct mail_index index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1663 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1664 return index->cache;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1665 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1666
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1667 int mail_index_set_error(struct mail_index index, const char fmt, ...)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1668 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1669 va_list va;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1670
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1671 i_free(index->error);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1672
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1673 if (fmt == NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1674 index->error = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1675 else {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1676 va_start(va, fmt);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1677 index->error = i_strdup_vprintf(fmt, va);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1678 va_end(va);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1679
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1680 i_error("%s", index->error);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1681 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1682
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1683 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1684 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1685
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1686 void mail_index_set_inconsistent(struct mail_index *index)
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1687 {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1688 index->indexid = 0;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1689 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1690
1945 268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1691 void mail_index_mark_corrupted(struct mail_index *index)
1943 e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1692 {
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1693 struct mail_index_header hdr;
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1694
1945 268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1695 mail_index_set_inconsistent(index);
268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1696
1943 e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1697 if (index->readonly)
1945 268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1698 return;
1943 e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1699
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1700 hdr = *index->hdr;
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1701 hdr.flags \|= MAIL_INDEX_HDR_FLAG_CORRUPTED;
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1702 if (mail_index_write_base_header(index, &hdr) == 0) {
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1703 if (!MAIL_INDEX_IS_IN_MEMORY(index) && fsync(index->fd) < 0)
1945 268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1704 mail_index_set_syscall_error(index, "fsync()");
268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1705 }
1943 e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1706 }
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1707
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1708 int mail_index_set_syscall_error(struct mail_index *index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1709 const char *function)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1710 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1711 i_assert(function != NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1712
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1713 if (ENOSPACE(errno)) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1714 index->nodiskspace = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1715 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1716 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1717
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1718 return mail_index_set_error(index, "%s failed with index file %s: %m",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1719 function, index->filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1720 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1721
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1722 int mail_index_file_set_syscall_error(struct mail_index *index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1723 const char *filepath,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1724 const char *function)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1725 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1726 i_assert(filepath != NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1727 i_assert(function != NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1728
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1729 if (ENOSPACE(errno)) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1730 index->nodiskspace = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1731 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1732 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1733
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1734 return mail_index_set_error(index, "%s failed with file %s: %m",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1735 function, filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1736 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1737
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1738 enum mail_index_error mail_index_get_last_error(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1739 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1740 if (index->nodiskspace)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1741 return MAIL_INDEX_ERROR_DISKSPACE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1742 if (index->error != NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1743 return MAIL_INDEX_ERROR_INTERNAL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1744
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1745 return MAIL_INDEX_ERROR_NONE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1746 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1747
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1748 const char mail_index_get_error_message(struct mail_index index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1749 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1750 return index->error;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1751 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1752
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1753 void mail_index_reset_error(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1754 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1755 if (index->error != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1756 i_free(index->error);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1757 index->error = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1758 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1759
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1760 index->nodiskspace = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1761 index->index_lock_timeout = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1762 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1763
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1764 uint32_t mail_index_uint32_to_offset(uint32_t offset)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1765 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1766 unsigned char buf[4];
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1767
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1768 i_assert(offset < 0x40000000);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1769 i_assert((offset & 3) == 0);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1770
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1771 offset >>= 2;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1772 buf[0] = 0x80 \| ((offset & 0x0fe00000) >> 21);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1773 buf[1] = 0x80 \| ((offset & 0x001fc000) >> 14);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1774 buf[2] = 0x80 \| ((offset & 0x00003f80) >> 7);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1775 buf[3] = 0x80 \| (offset & 0x0000007f);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1776 return ((uint32_t ) buf);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1777 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1778
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1779 uint32_t mail_index_offset_to_uint32(uint32_t offset)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1780 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1781 const unsigned char buf = (const unsigned char ) &offset;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1782
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1783 if ((offset & 0x80808080) != 0x80808080)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1784 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1785
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1786 return (((uint32_t)buf[3] & 0x7f) << 2) \|
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1787 (((uint32_t)buf[2] & 0x7f) << 9) \|
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1788 (((uint32_t)buf[1] & 0x7f) << 16) \|
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1789 (((uint32_t)buf[0] & 0x7f) << 23);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	1790 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-index.c @ 3582:bfa542369fdc HEAD