dovecot/core-2.2: src/lib-index/mail-index.c annotate

annotate src/lib-index/mail-index.c @ 5453:a0a1432bb4e8 HEAD

Better type safety to module_contexts arrays. Already fixed some bugs.

author	Timo Sirainen <tss@iki.fi>
date	Thu, 29 Mar 2007 17:19:47 +0300
parents	c8bb83980ff7
children	0d8f6bfb9fdb

rev	line source
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1 /* Copyright (C) 2003-2004 Timo Sirainen */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
3470 346a494c2feb Moved array declaration to array-decl.h and include it in lib.h. So array.h Timo Sirainen <tss@iki.fi> parents: 3445 diff changeset	4 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "buffer.h"
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	6 #include "hash.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	7 #include "mmap-util.h"
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4067 diff changeset	8 #include "nfs-workarounds.h"
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	9 #include "read-full.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	10 #include "write-full.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	11 #include "mail-index-private.h"
5272 82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	12 #include "mail-index-view-private.h"
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	13 #include "mail-index-sync-private.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	14 #include "mail-transaction-log.h"
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	15 #include "mail-cache.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	16
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	17 #include <stdio.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	18 #include <stddef.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	19 #include <time.h>
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	20 #include <sys/stat.h>
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	21
5453 a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs. Timo Sirainen <tss@iki.fi> parents: 5430 diff changeset	22 struct mail_index_module_register mail_index_module_register = { 0 };
4453 85fcdb478608 Beginnings of joining mail-storage API more closely to mail-index, so that Timo Sirainen <timo.sirainen@movial.fi> parents: 4451 diff changeset	23
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	24 static int mail_index_try_open_only(struct mail_index *index);
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	25 static void mail_index_create_in_memory(struct mail_index *index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	26 const struct mail_index_header *hdr);
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	27
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	28 struct mail_index mail_index_alloc(const char dir, const char *prefix)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	29 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	30 struct mail_index *index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	31
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	32 index = i_new(struct mail_index, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	33 index->dir = i_strdup(dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	34 index->prefix = i_strdup(prefix);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	35 index->fd = -1;
2116 fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	36
5028 774c965ac79a Memory pool size changes Timo Sirainen <tss@iki.fi> parents: 5024 diff changeset	37 index->extension_pool =
774c965ac79a Memory pool size changes Timo Sirainen <tss@iki.fi> parents: 5024 diff changeset	38 pool_alloconly_create(MEMPOOL_GROWING"index extension", 1024);
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	39 p_array_init(&index->extensions, index->extension_pool, 5);
bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	40 i_array_init(&index->sync_lost_handlers, 4);
5453 a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs. Timo Sirainen <tss@iki.fi> parents: 5430 diff changeset	41 i_array_init(&index->module_contexts,
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs. Timo Sirainen <tss@iki.fi> parents: 5430 diff changeset	42 I_MIN(5, mail_index_module_register.id));
4453 85fcdb478608 Beginnings of joining mail-storage API more closely to mail-index, so that Timo Sirainen <timo.sirainen@movial.fi> parents: 4451 diff changeset	43
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	44 index->mode = 0600;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	45 index->gid = (gid_t)-1;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	46
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	47 index->keywords_ext_id =
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	48 mail_index_ext_register(index, "keywords", 128, 2, 1);
3086 66c44404f9a1 Keyword fixes. Timo Sirainen <tss@iki.fi> parents: 3016 diff changeset	49 index->keywords_pool = pool_alloconly_create("keywords", 512);
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	50 i_array_init(&index->keywords, 16);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	51 index->keywords_hash =
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	52 hash_create(default_pool, index->keywords_pool, 0,
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	53 strcase_hash, (hash_cmp_callback_t *)strcasecmp);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	54 return index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	55 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	56
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	57 void mail_index_free(struct mail_index **_index)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	59 struct mail_index index = _index;
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	60
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	61 *_index = NULL;
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1965 diff changeset	62 mail_index_close(index);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	63
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	64 hash_destroy(index->keywords_hash);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	65 pool_unref(index->extension_pool);
3086 66c44404f9a1 Keyword fixes. Timo Sirainen <tss@iki.fi> parents: 3016 diff changeset	66 pool_unref(index->keywords_pool);
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1965 diff changeset	67
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	68 array_free(&index->sync_lost_handlers);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	69 array_free(&index->keywords);
5453 a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs. Timo Sirainen <tss@iki.fi> parents: 5430 diff changeset	70 array_free(&index->module_contexts);
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	71
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	72 i_free(index->error);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	73 i_free(index->dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	74 i_free(index->prefix);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	75 i_free(index);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	76 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	77
2650 9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	78 void mail_index_set_permissions(struct mail_index *index,
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	79 mode_t mode, gid_t gid)
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	80 {
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	81 index->mode = mode & 0666;
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	82 index->gid = gid;
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	83 }
9b78674428f5 Get index file mode and GID from dovecot-shared file. Timo Sirainen <tss@iki.fi> parents: 2618 diff changeset	84
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	85 uint32_t mail_index_ext_register(struct mail_index index, const char name,
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	86 uint32_t default_hdr_size,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	87 uint16_t default_record_size,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	88 uint16_t default_record_align)
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	89 {
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	90 const struct mail_index_registered_ext *extensions;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	91 struct mail_index_registered_ext rext;
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	92 unsigned int i, ext_count;
2115 f720b3f15333 Modifying extra_records should work now. Timo Sirainen <tss@iki.fi> parents: 2114 diff changeset	93
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	94 extensions = array_get(&index->extensions, &ext_count);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	95
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	96 /* see if it's already there */
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	97 for (i = 0; i < ext_count; i++) {
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	98 if (strcmp(extensions[i].name, name) == 0)
2116 fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	99 return i;
fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	100 }
fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	101
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	102 memset(&rext, 0, sizeof(rext));
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	103 rext.name = p_strdup(index->extension_pool, name);
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	104 rext.index_idx = ext_count;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	105 rext.hdr_size = default_hdr_size;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	106 rext.record_size = default_record_size;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	107 rext.record_align = default_record_align;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	108
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	109 array_append(&index->extensions, &rext, 1);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	110 return ext_count;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	111 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	112
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	113 void mail_index_register_expunge_handler(struct mail_index *index,
4903 204d7edc7cdc Added context parameter type safety checks for most callback APIs. Timo Sirainen <tss@iki.fi> parents: 4876 diff changeset	114 uint32_t ext_id, bool call_always,
4490 56d17f2e1d66 Added support for calling expunge handler for an extension even if the Timo Sirainen <tss@iki.fi> parents: 4453 diff changeset	115 mail_index_expunge_handler_t *cb,
4903 204d7edc7cdc Added context parameter type safety checks for most callback APIs. Timo Sirainen <tss@iki.fi> parents: 4876 diff changeset	116 void *context)
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	117 {
3817 dc560ef2654b Moved expunge handlers to mail_index_registered_ext. Timo Sirainen <tss@iki.fi> parents: 3816 diff changeset	118 struct mail_index_registered_ext *rext;
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	119
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	120 rext = array_idx_modifiable(&index->extensions, ext_id);
4616 b0966236fda7 Memory leak fixes Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	121 i_assert(rext->expunge_handler == NULL \|\| rext->expunge_handler == cb);
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	122
3817 dc560ef2654b Moved expunge handlers to mail_index_registered_ext. Timo Sirainen <tss@iki.fi> parents: 3816 diff changeset	123 rext->expunge_handler = cb;
4490 56d17f2e1d66 Added support for calling expunge handler for an extension even if the Timo Sirainen <tss@iki.fi> parents: 4453 diff changeset	124 rext->expunge_context = context;
56d17f2e1d66 Added support for calling expunge handler for an extension even if the Timo Sirainen <tss@iki.fi> parents: 4453 diff changeset	125 rext->expunge_handler_call_always = call_always;
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	126 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	127
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	128 void mail_index_unregister_expunge_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	129 uint32_t ext_id)
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	130 {
3817 dc560ef2654b Moved expunge handlers to mail_index_registered_ext. Timo Sirainen <tss@iki.fi> parents: 3816 diff changeset	131 struct mail_index_registered_ext *rext;
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	132
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	133 rext = array_idx_modifiable(&index->extensions, ext_id);
3817 dc560ef2654b Moved expunge handlers to mail_index_registered_ext. Timo Sirainen <tss@iki.fi> parents: 3816 diff changeset	134 i_assert(rext->expunge_handler != NULL);
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	135
3817 dc560ef2654b Moved expunge handlers to mail_index_registered_ext. Timo Sirainen <tss@iki.fi> parents: 3816 diff changeset	136 rext->expunge_handler = NULL;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	137 }
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	138
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	139 void mail_index_register_sync_handler(struct mail_index *index, uint32_t ext_id,
2869 d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2857 diff changeset	140 mail_index_sync_handler_t *cb,
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2857 diff changeset	141 enum mail_index_sync_handler_type type)
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	142 {
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	143 struct mail_index_registered_ext *rext;
2869 d2daf90c79f1 Cache invalidating needed to hook into view syncing as well. Timo Sirainen <tss@iki.fi> parents: 2857 diff changeset	144
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	145 rext = array_idx_modifiable(&index->extensions, ext_id);
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	146 i_assert(rext->sync_handler.callback == NULL);
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	147
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	148 rext->sync_handler.callback = cb;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	149 rext->sync_handler.type = type;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	150 }
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	151
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	152 void mail_index_unregister_sync_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	153 uint32_t ext_id)
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	154 {
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	155 struct mail_index_registered_ext *rext;
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	156
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	157 rext = array_idx_modifiable(&index->extensions, ext_id);
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	158 i_assert(rext->sync_handler.callback != NULL);
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	159
3816 575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	160 rext->sync_handler.callback = NULL;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for Timo Sirainen <tss@iki.fi> parents: 3582 diff changeset	161 rext->sync_handler.type = 0;
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	162 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	163
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	164 void mail_index_register_sync_lost_handler(struct mail_index *index,
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	165 mail_index_sync_lost_handler_t *cb)
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	166 {
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	167 array_append(&index->sync_lost_handlers, &cb, 1);
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	168 }
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	169
3224 f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	170 void mail_index_unregister_sync_lost_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	171 mail_index_sync_lost_handler_t *cb)
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	172 {
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	173 mail_index_sync_lost_handler_t const handlers;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	174 unsigned int i, count;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	175
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	176 handlers = array_get(&index->sync_lost_handlers, &count);
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	177 for (i = 0; i < count; i++) {
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	178 if (handlers[i] == cb) {
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	179 array_delete(&index->sync_lost_handlers, i, 1);
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	180 break;
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	181 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	182 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	183 }
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes Timo Sirainen <tss@iki.fi> parents: 3222 diff changeset	184
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	185 static void mail_index_map_init_extbufs(struct mail_index_map *map,
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	186 unsigned int initial_count)
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	187 {
3294 4600e247da86 Raised initial pool sizes. Timo Sirainen <tss@iki.fi> parents: 3254 diff changeset	188 #define EXTENSION_NAME_APPROX_LEN 20
4495 bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	189 #define EXT_GLOBAL_ALLOC_SIZE \
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	190 ((sizeof(map->extensions) + BUFFER_APPROX_SIZE) * 2)
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	191 #define EXT_PER_ALLOC_SIZE \
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	192 (EXTENSION_NAME_APPROX_LEN + \
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	193 sizeof(struct mail_index_ext) + sizeof(uint32_t))
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	194 size_t size;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	195
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	196 if (map->extension_pool == NULL) {
4495 bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	197 size = EXT_GLOBAL_ALLOC_SIZE +
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	198 initial_count * EXT_PER_ALLOC_SIZE;
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	199 map->extension_pool =
5028 774c965ac79a Memory pool size changes Timo Sirainen <tss@iki.fi> parents: 5024 diff changeset	200 pool_alloconly_create("map extensions",
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	201 nearest_power(size));
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	202 } else {
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	203 p_clear(map->extension_pool);
4495 bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	204
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	205 /* try to use the existing pool's size for initial_count so
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	206 we don't grow it unneededly */
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	207 size = p_get_max_easy_alloc_size(map->extension_pool);
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	208 if (size > EXT_GLOBAL_ALLOC_SIZE + EXT_PER_ALLOC_SIZE) {
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	209 initial_count = (size - EXT_GLOBAL_ALLOC_SIZE) /
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	210 EXT_PER_ALLOC_SIZE;
bcc5d4dc5400 Small memory usage optimization. Timo Sirainen <tss@iki.fi> parents: 4490 diff changeset	211 }
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	212 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	213
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	214 p_array_init(&map->extensions, map->extension_pool, initial_count);
bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	215 p_array_init(&map->ext_id_map, map->extension_pool, initial_count);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	216 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	217
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	218 uint32_t mail_index_map_lookup_ext(struct mail_index_map map, const char name)
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	219 {
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	220 const struct mail_index_ext *extensions;
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	221 unsigned int i, size;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	222
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	223 if (!array_is_created(&map->extensions))
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	224 return (uint32_t)-1;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	225
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	226 extensions = array_get(&map->extensions, &size);
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	227 for (i = 0; i < size; i++) {
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	228 if (strcmp(extensions[i].name, name) == 0)
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	229 return i;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	230 }
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	231 return (uint32_t)-1;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	232 }
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	233
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	234 uint32_t
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	235 mail_index_map_register_ext(struct mail_index *index,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	236 struct mail_index_map map, const char name,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	237 uint32_t hdr_offset, uint32_t hdr_size,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	238 uint32_t record_offset, uint32_t record_size,
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	239 uint32_t record_align, uint32_t reset_id)
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	240 {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	241 struct mail_index_ext *ext;
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	242 uint32_t idx, empty_idx = (uint32_t)-1;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	243
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	244 if (!array_is_created(&map->extensions)) {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	245 mail_index_map_init_extbufs(map, 5);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	246 idx = 0;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	247 } else {
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	248 idx = array_count(&map->extensions);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	249 }
3147 6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	250 i_assert(mail_index_map_lookup_ext(map, name) == (uint32_t)-1);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	251
3445 74475a126e2f Renamed array_modifyable_() functions. I could never remember how they were Timo Sirainen <tss@iki.fi>* parents: 3369 diff changeset	252 ext = array_append_space(&map->extensions);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	253 ext->name = p_strdup(map->extension_pool, name);
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	254 ext->hdr_offset = hdr_offset;
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	255 ext->hdr_size = hdr_size;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	256 ext->record_offset = record_offset;
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	257 ext->record_size = record_size;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	258 ext->record_align = record_align;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	259 ext->reset_id = reset_id;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	260
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	261 ext->index_idx = mail_index_ext_register(index, name, hdr_size,
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	262 record_size, record_align);
2921 a5eda271256f Wrong extension sync_handler might have been called if index extensions Timo Sirainen <tss@iki.fi> parents: 2913 diff changeset	263
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	264 /* Update index ext_id -> map ext_id mapping. Fill non-used
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	265 ext_ids with (uint32_t)-1 */
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	266 while (array_count(&map->ext_id_map) < ext->index_idx)
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	267 array_append(&map->ext_id_map, &empty_idx, 1);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	268 array_idx_set(&map->ext_id_map, ext->index_idx, &idx);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	269 return idx;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	270 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	271
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	272 static bool size_check(size_t *size_left, size_t size)
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	273 {
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	274 if (size > *size_left)
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	275 return FALSE;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	276 *size_left -= size;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	277 return TRUE;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	278 }
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	279
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	280 static size_t get_align(size_t name_len)
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	281 {
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	282 size_t size = sizeof(struct mail_index_ext_header) + name_len;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	283 return MAIL_INDEX_HEADER_SIZE_ALIGN(size) - size;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	284 }
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	285
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	286 static int mail_index_parse_extensions(struct mail_index *index,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	287 struct mail_index_map *map)
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	288 {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	289 const struct mail_index_ext_header *ext_hdr;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	290 unsigned int i, old_count;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	291 const char *name;
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	292 uint32_t ext_id, offset, name_offset;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	293 size_t size_left;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	294
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	295 /* extension headers always start from 64bit offsets, so if base header
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	296 doesn't happen to be 64bit aligned we'll skip some bytes */
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	297 offset = MAIL_INDEX_HEADER_SIZE_ALIGN(map->hdr.base_header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	298 if (offset >= map->hdr.header_size && map->extension_pool == NULL) {
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	299 /* nothing to do, skip allocatations and all */
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	300 return 1;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	301 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	302
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	303 old_count = array_count(&index->extensions);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	304 mail_index_map_init_extbufs(map, old_count + 5);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	305
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	306 ext_id = (uint32_t)-1;
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	307 for (i = 0; i < old_count; i++)
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	308 array_append(&map->ext_id_map, &ext_id, 1);
2116 fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	309
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	310 while (offset < map->hdr.header_size) {
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	311 ext_hdr = CONST_PTR_OFFSET(map->hdr_base, offset);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	312
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	313 /* Extension header contains:
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	314 - struct mail_index_ext_header
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	315 - name (not 0-terminated)
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	316 - 64bit alignment padding
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	317 - extension header contents
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	318 - 64bit alignment padding
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	319 */
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	320 size_left = map->hdr.header_size - offset;
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	321 if (!size_check(&size_left, sizeof(*ext_hdr)) \|\|
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	322 !size_check(&size_left, ext_hdr->name_size) \|\|
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	323 !size_check(&size_left, get_align(ext_hdr->name_size)) \|\|
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	324 !size_check(&size_left, ext_hdr->hdr_size)) {
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	325 mail_index_set_error(index, "Corrupted index file %s: "
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	326 "Header extension goes outside header",
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	327 index->filepath);
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	328 return -1;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	329 }
3147 6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	330
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	331 offset += sizeof(*ext_hdr);
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	332 name_offset = offset;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	333 offset += ext_hdr->name_size + get_align(ext_hdr->name_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	334
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	335 t_push();
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	336 name = t_strndup(CONST_PTR_OFFSET(map->hdr_base, name_offset),
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	337 ext_hdr->name_size);
3147 6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	338
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	339 if (mail_index_map_lookup_ext(map, name) != (uint32_t)-1) {
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	340 mail_index_set_error(index, "Corrupted index file %s: "
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	341 "Duplicate header extension %s",
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	342 index->filepath, name);
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	343 t_pop();
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	344 return -1;
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	345 }
6c7d463eaf1e Don't allow duplicate extension names in index header. Timo Sirainen <tss@iki.fi> parents: 3129 diff changeset	346
5011 257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	347 if (map->hdr.record_size <
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	348 ext_hdr->record_offset + ext_hdr->record_size) {
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	349 mail_index_set_error(index, "Corrupted index file %s: "
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	350 "Record field %s points outside record size "
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	351 "(%u < %u+%u)", index->filepath, name,
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	352 map->hdr.record_size,
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	353 ext_hdr->record_offset, ext_hdr->record_size);
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	354 t_pop();
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	355 return -1;
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	356 }
257512aaa8f3 Make sure that extensions don't point outside allocated record size. Timo Sirainen <tss@iki.fi> parents: 5008 diff changeset	357
3497 872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	358 if ((ext_hdr->record_offset % ext_hdr->record_align) != 0 \|\|
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	359 (map->hdr.record_size % ext_hdr->record_align) != 0) {
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	360 mail_index_set_error(index, "Corrupted index file %s: "
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	361 "Record field %s alignmentation %u not used",
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	362 index->filepath, name, ext_hdr->record_align);
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	363 t_pop();
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	364 return -1;
872218271484 Check that record alignmentation is actually used correctly. If not, Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	365 }
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	366 mail_index_map_register_ext(index, map, name,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	367 offset, ext_hdr->hdr_size,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	368 ext_hdr->record_offset,
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	369 ext_hdr->record_size,
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	370 ext_hdr->record_align,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	371 ext_hdr->reset_id);
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	372 t_pop();
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	373
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	374 offset += MAIL_INDEX_HEADER_SIZE_ALIGN(ext_hdr->hdr_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	375 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	376 return 1;
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	377 }
4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	378
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	379 bool mail_index_keyword_lookup(struct mail_index *index,
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	380 const char *keyword, bool autocreate,
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	381 unsigned int *idx_r)
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	382 {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	383 char *keyword_dup;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	384 void *value;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	385
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	386 /* keywords_hash keeps a name => index mapping of keywords.
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	387 Keywords are never removed from it, so the index values are valid
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	388 for the lifetime of the mail_index. */
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	389 if (hash_lookup_full(index->keywords_hash, keyword, NULL, &value)) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	390 *idx_r = POINTER_CAST_TO(value, unsigned int);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	391 return TRUE;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	392 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	393
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	394 if (!autocreate) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	395 *idx_r = (unsigned int)-1;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	396 return FALSE;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	397 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	398
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	399 keyword = keyword_dup = p_strdup(index->keywords_pool, keyword);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	400 *idx_r = array_count(&index->keywords);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	401
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	402 hash_insert(index->keywords_hash, keyword_dup, POINTER_CAST(*idx_r));
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	403 array_append(&index->keywords, &keyword, 1);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	404 return TRUE;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	405 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	406
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	407 int mail_index_map_parse_keywords(struct mail_index *index,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	408 struct mail_index_map *map)
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	409 {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	410 const struct mail_index_ext *ext;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	411 const struct mail_index_keyword_header *kw_hdr;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	412 const struct mail_index_keyword_header_rec *kw_rec;
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	413 const char *name;
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	414 unsigned int i, name_area_end_offset, old_count;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	415 uint32_t ext_id;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	416
5022 2c60e03d2bf6 Keywords were sometimes being re-added to the index even while they already Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	417 map->keywords_read = TRUE;
2c60e03d2bf6 Keywords were sometimes being re-added to the index even while they already Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	418
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	419 ext_id = mail_index_map_lookup_ext(map, "keywords");
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	420 if (ext_id == (uint32_t)-1) {
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	421 if (array_is_created(&map->keyword_idx_map))
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	422 array_clear(&map->keyword_idx_map);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	423 return 0;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	424 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	425
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	426 ext = array_idx(&map->extensions, ext_id);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	427
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	428 /* Extension header contains:
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	429 - struct mail_index_keyword_header
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	430 - struct mail_index_keyword_header_rec * keywords_count
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	431 - const char names[] * keywords_count
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	432 */
4220 2af75b0379bb Added assert Timo Sirainen <timo.sirainen@movial.fi> parents: 4072 diff changeset	433 i_assert(ext->hdr_offset < map->hdr.header_size);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	434 kw_hdr = CONST_PTR_OFFSET(map->hdr_base, ext->hdr_offset);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	435 kw_rec = (const void *)(kw_hdr + 1);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	436 name = (const char *)(kw_rec + kw_hdr->keywords_count);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	437
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	438 old_count = !array_is_created(&map->keyword_idx_map) ? 0 :
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	439 array_count(&map->keyword_idx_map);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	440
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	441 /* Keywords can only be added into same mapping. Removing requires a
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	442 new mapping (recreating the index file) */
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	443 if (kw_hdr->keywords_count == old_count) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	444 /* nothing changed */
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	445 return 0;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	446 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	447
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	448 /* make sure the header is valid */
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	449 if (kw_hdr->keywords_count < old_count) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	450 mail_index_set_error(index, "Corrupted index file %s: "
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	451 "Keywords removed unexpectedly",
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	452 index->filepath);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	453 return -1;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	454 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	455
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	456 if ((size_t)(name - (const char *)kw_hdr) > ext->hdr_size) {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	457 mail_index_set_error(index, "Corrupted index file %s: "
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	458 "keywords_count larger than header size",
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	459 index->filepath);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	460 return -1;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	461 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	462
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	463 name_area_end_offset = (const char *)kw_hdr + ext->hdr_size - name;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	464 for (i = 0; i < kw_hdr->keywords_count; i++) {
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	465 if (kw_rec[i].name_offset > name_area_end_offset) {
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	466 mail_index_set_error(index, "Corrupted index file %s: "
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	467 "name_offset points outside allocated header",
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	468 index->filepath);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	469 return -1;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	470 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	471 }
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	472 if (name[name_area_end_offset-1] != '\0') {
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	473 mail_index_set_error(index, "Corrupted index file %s: "
3550 6fbdd79b32a5 Error message update Timo Sirainen <tss@iki.fi> parents: 3497 diff changeset	474 "Keyword header doesn't end with NUL",
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	475 index->filepath);
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	476 return -1;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	477 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	478
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	479 /* create file -> index mapping */
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	480 if (!array_is_created(&map->keyword_idx_map))
bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	481 i_array_init(&map->keyword_idx_map, kw_hdr->keywords_count);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	482
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	483 #ifdef DEBUG
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	484 /* Check that existing headers are still the same. It's behind DEBUG
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	485 since it's pretty useless waste of CPU normally. */
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	486 for (i = 0; i < array_count(&map->keyword_idx_map); i++) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	487 const char *keyword = name + kw_rec[i].name_offset;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	488 const unsigned int *old_idx;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	489 unsigned int idx;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	490
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	491 old_idx = array_idx(&map->keyword_idx_map, i);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	492 if (!mail_index_keyword_lookup(index, keyword, FALSE, &idx) \|\|
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	493 idx != *old_idx) {
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	494 mail_index_set_error(index, "Corrupted index file %s: "
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	495 "Keywords changed unexpectedly",
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	496 index->filepath);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	497 return -1;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	498 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	499 }
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	500 #endif
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	501 /* Register the newly seen keywords */
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	502 i = array_count(&map->keyword_idx_map);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	503 for (; i < kw_hdr->keywords_count; i++) {
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	504 const char *keyword = name + kw_rec[i].name_offset;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	505 unsigned int idx;
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	506
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	507 (void)mail_index_keyword_lookup(index, keyword, TRUE, &idx);
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	508 array_append(&map->keyword_idx_map, &idx, 1);
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	509 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	510 return 0;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	511 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	512
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	513 const ARRAY_TYPE(keywords) mail_index_get_keywords(struct mail_index index)
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	514 {
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	515 /* Make sure all the keywords are in index->keywords. It's quick to do
6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	516 if nothing has changed. */
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	517 (void)mail_index_map_parse_keywords(index, index->map);
3818 6758c0f3c318 Comment updates and minor cleanups Timo Sirainen <tss@iki.fi> parents: 3817 diff changeset	518
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	519 return &index->keywords;
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	520 }
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2956 diff changeset	521
5212 0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	522 static bool mail_index_check_header_compat(const struct mail_index_header *hdr)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	523 {
3819 61f0e1fe9b55 Moved uoff_t and time_t compatibility checks to cache file, since they're Timo Sirainen <tss@iki.fi> parents: 3818 diff changeset	524 enum mail_index_header_compat_flags compat_flags = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	525
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	526 #ifndef WORDS_BIGENDIAN
3819 61f0e1fe9b55 Moved uoff_t and time_t compatibility checks to cache file, since they're Timo Sirainen <tss@iki.fi> parents: 3818 diff changeset	527 compat_flags \|= MAIL_INDEX_COMPAT_LITTLE_ENDIAN;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	528 #endif
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	529
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	530 if (hdr->major_version != MAIL_INDEX_MAJOR_VERSION) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	531 /* major version change - handle silently(?) */
5212 0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	532 return FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	533 }
3819 61f0e1fe9b55 Moved uoff_t and time_t compatibility checks to cache file, since they're Timo Sirainen <tss@iki.fi> parents: 3818 diff changeset	534 if (hdr->compat_flags != compat_flags) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	535 /* architecture change - handle silently(?) */
5212 0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	536 return FALSE;
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	537 }
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	538
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	539 if ((hdr->flags & MAIL_INDEX_HDR_FLAG_CORRUPTED) != 0) {
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	540 /* we've already complained about it */
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	541 return FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	542 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	543
5212 0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	544 return TRUE;
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	545 }
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	546
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	547 static int mail_index_check_header(struct mail_index *index,
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	548 struct mail_index_map *map)
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	549 {
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	550 const struct mail_index_header *hdr = &map->hdr;
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	551
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	552 if (!mail_index_check_header_compat(hdr))
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	553 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	554
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	555 /* following some extra checks that only take a bit of CPU */
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	556 if (hdr->uid_validity == 0 && hdr->next_uid != 1) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	557 mail_index_set_error(index, "Corrupted index file %s: "
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	558 "uid_validity = 0, next_uid = %u",
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	559 index->filepath, hdr->next_uid);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	560 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	561 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	562
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	563 if (hdr->record_size < sizeof(struct mail_index_record)) {
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	564 mail_index_set_error(index, "Corrupted index file %s: "
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	565 "record_size too small: %u < %"PRIuSIZE_T,
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	566 index->filepath, hdr->record_size,
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	567 sizeof(struct mail_index_record));
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	568 return -1;
4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	569 }
4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	570
3353 286cc0cd888a Check broken flag counter values every time when updating them. Added Timo Sirainen <tss@iki.fi> parents: 3296 diff changeset	571 if ((hdr->flags & MAIL_INDEX_HDR_FLAG_FSCK) != 0)
286cc0cd888a Check broken flag counter values every time when updating them. Added Timo Sirainen <tss@iki.fi> parents: 3296 diff changeset	572 return 0;
286cc0cd888a Check broken flag counter values every time when updating them. Added Timo Sirainen <tss@iki.fi> parents: 3296 diff changeset	573
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	574 if (hdr->next_uid == 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	575 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	576
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2021 diff changeset	577 if (hdr->recent_messages_count > hdr->messages_count \|\|
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2021 diff changeset	578 hdr->seen_messages_count > hdr->messages_count \|\|
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	579 hdr->deleted_messages_count > hdr->messages_count)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	580 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	581 if (hdr->first_recent_uid_lowwater > hdr->next_uid \|\|
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	582 hdr->first_unseen_uid_lowwater > hdr->next_uid \|\|
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	583 hdr->first_deleted_uid_lowwater > hdr->next_uid)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	584 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	585
4775 22b11dd2da09 When verifying the header, check that next_uid is larger than last message's Timo Sirainen <tss@iki.fi> parents: 4662 diff changeset	586 if (map->records_count > 0) {
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	587 /* last message's UID must be smaller than next_uid.
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	588 also make sure it's not zero. */
4775 22b11dd2da09 When verifying the header, check that next_uid is larger than last message's Timo Sirainen <tss@iki.fi> parents: 4662 diff changeset	589 const struct mail_index_record *rec;
22b11dd2da09 When verifying the header, check that next_uid is larger than last message's Timo Sirainen <tss@iki.fi> parents: 4662 diff changeset	590
22b11dd2da09 When verifying the header, check that next_uid is larger than last message's Timo Sirainen <tss@iki.fi> parents: 4662 diff changeset	591 rec = MAIL_INDEX_MAP_IDX(map, map->records_count-1);
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	592 if (rec->uid == 0 \|\| rec->uid >= hdr->next_uid)
4775 22b11dd2da09 When verifying the header, check that next_uid is larger than last message's Timo Sirainen <tss@iki.fi> parents: 4662 diff changeset	593 return 0;
22b11dd2da09 When verifying the header, check that next_uid is larger than last message's Timo Sirainen <tss@iki.fi> parents: 4662 diff changeset	594 }
22b11dd2da09 When verifying the header, check that next_uid is larger than last message's Timo Sirainen <tss@iki.fi> parents: 4662 diff changeset	595
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	596 return mail_index_parse_extensions(index, map);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	597 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	598
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	599 static void mail_index_map_clear(struct mail_index *index,
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	600 struct mail_index_map *map)
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	601 {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	602 if (map->buffer != NULL) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	603 i_assert(map->mmap_base == NULL);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	604 buffer_free(map->buffer);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	605 map->buffer = NULL;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	606 } else if (map->mmap_base != NULL) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	607 i_assert(map->buffer == NULL);
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	608 if (munmap(map->mmap_base, map->mmap_size) < 0)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	609 mail_index_set_syscall_error(index, "munmap()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	610 map->mmap_base = NULL;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	611 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	612
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	613 if (map->refcount > 0) {
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	614 memset(&map->hdr, 0, sizeof(map->hdr));
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	615 map->mmap_size = 0;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	616 map->mmap_used_size = 0;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	617 map->records = NULL;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	618 map->records_count = 0;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	619 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	620 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	621
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	622 void mail_index_unmap(struct mail_index index, struct mail_index_map *_map)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	623 {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	624 struct mail_index_map map = _map;
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	625
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	626 *_map = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	627 if (--map->refcount > 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	628 return;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	629
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	630 i_assert(map->refcount == 0);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	631 mail_index_map_clear(index, map);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	632 if (map->extension_pool != NULL)
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	633 pool_unref(map->extension_pool);
3254 a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	634 if (array_is_created(&map->keyword_idx_map))
a2943c050571 Keywords are now stored in X-Keywords headers in mbox. Did several related Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	635 array_free(&map->keyword_idx_map);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	636 buffer_free(map->hdr_copy_buf);
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	637 i_free(map);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	638 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	639
2922 fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	640 static void mail_index_map_copy_hdr(struct mail_index_map *map,
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	641 const struct mail_index_header *hdr)
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	642 {
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	643 if (hdr->base_header_size < sizeof(map->hdr)) {
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	644 /* header smaller than ours, make a copy so our newer headers
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	645 won't have garbage in them */
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	646 memset(&map->hdr, 0, sizeof(map->hdr));
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	647 memcpy(&map->hdr, hdr, hdr->base_header_size);
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	648 } else {
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	649 map->hdr = *hdr;
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	650 }
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	651 }
fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	652
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	653 static int mail_index_mmap(struct mail_index index, struct mail_index_map map)
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	654 {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	655 const struct mail_index_header *hdr;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	656 unsigned int records_count;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	657
2193 1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	658 i_assert(!map->write_to_disk);
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	659
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	660 if (map->buffer != NULL) {
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	661 /* we had temporarily used a buffer, eg. for updating index */
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	662 buffer_free(map->buffer);
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	663 map->buffer = NULL;
1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	664 }
2043 4548def7ac02 Don't try to re-map dirty memory mapping. Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	665
2761 f96a7b2d1c52 Removed mmap() memory area protections. They weren't always set correctly. Timo Sirainen <tss@iki.fi> parents: 2723 diff changeset	666 map->mmap_base = index->readonly ?
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	667 mmap_ro_file(index->fd, &map->mmap_size) :
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	668 mmap_rw_file(index->fd, &map->mmap_size);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	669 if (map->mmap_base == MAP_FAILED) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	670 map->mmap_base = NULL;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	671 mail_index_set_syscall_error(index, "mmap()");
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	672 return -1;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	673 }
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	674
2822 280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	675 hdr = map->mmap_base;
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	676 if (map->mmap_size >
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	677 offsetof(struct mail_index_header, major_version) &&
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	678 hdr->major_version != MAIL_INDEX_MAJOR_VERSION) {
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	679 /* major version change - handle silently */
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	680 return 0;
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	681 }
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	682
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	683 if (map->mmap_size < MAIL_INDEX_HEADER_MIN_SIZE) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	684 mail_index_set_error(index, "Corrupted index file %s: "
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	685 "File too small (%"PRIuSIZE_T")",
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	686 index->filepath, map->mmap_size);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	687 return 0;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	688 }
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	689
5212 0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	690 if (!mail_index_check_header_compat(hdr)) {
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	691 /* Can't use this file */
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	692 return 0;
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	693 }
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	694
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	695 map->mmap_used_size = hdr->header_size +
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	696 hdr->messages_count * hdr->record_size;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	697
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	698 if (map->mmap_used_size > map->mmap_size) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	699 records_count = (map->mmap_size - hdr->header_size) /
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	700 hdr->record_size;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	701 mail_index_set_error(index, "Corrupted index file %s: "
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	702 "messages_count too large (%u > %u)",
2122 db5807f350ca crashfix on error Timo Sirainen <tss@iki.fi> parents: 2116 diff changeset	703 index->filepath, hdr->messages_count,
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	704 records_count);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	705 return 0;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	706 }
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	707
2922 fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	708 mail_index_map_copy_hdr(map, hdr);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	709
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	710 map->hdr_base = map->mmap_base;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	711 map->records = PTR_OFFSET(map->mmap_base, map->hdr.header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	712 map->records_count = map->hdr.messages_count;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	713 return 1;
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	714 }
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	715
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	716 static int mail_index_read_header(struct mail_index *index,
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	717 void buf, size_t buf_size, size_t pos_r)
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	718 {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	719 size_t pos;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	720 int ret;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	721
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	722 memset(buf, 0, sizeof(struct mail_index_header));
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	723
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	724 /* try to read the whole header, but it's not necessarily an error to
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	725 read less since the older versions of the index format could be
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	726 smaller. Request reading up to buf_size, but accept if we only got
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	727 the header. */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	728 pos = 0;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	729 do {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	730 ret = pread(index->fd, PTR_OFFSET(buf, pos),
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	731 buf_size - pos, pos);
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	732 if (ret > 0)
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	733 pos += ret;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	734 } while (ret > 0 && pos < sizeof(struct mail_index_header));
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	735
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	736 *pos_r = pos;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	737 return ret;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	738 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	739
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	740 static int
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	741 mail_index_read_map(struct mail_index index, struct mail_index_map map,
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	742 bool *retry_r, bool try_retry)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	743 {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	744 const struct mail_index_header *hdr;
5058 2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	745 struct stat st;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	746 unsigned char buf[512];
1939 4698c121bae3 changes Timo Sirainen <tss@iki.fi> parents: 1935 diff changeset	747 void *data = NULL;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	748 ssize_t ret;
5059 baa0e9f60a24 cleanups Timo Sirainen <tss@iki.fi> parents: 5058 diff changeset	749 size_t pos, records_size;
baa0e9f60a24 cleanups Timo Sirainen <tss@iki.fi> parents: 5058 diff changeset	750 unsigned int records_count;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	751
2043 4548def7ac02 Don't try to re-map dirty memory mapping. Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	752 i_assert(map->mmap_base == NULL);
4548def7ac02 Don't try to re-map dirty memory mapping. Timo Sirainen <tss@iki.fi> parents: 2038 diff changeset	753
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	754 *retry_r = FALSE;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	755 ret = mail_index_read_header(index, buf, sizeof(buf), &pos);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	756 hdr = (const struct mail_index_header *)buf;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	757
2825 645b8442b519 one more major version change fix Timo Sirainen <tss@iki.fi> parents: 2824 diff changeset	758 if (pos > (ssize_t)offsetof(struct mail_index_header, major_version) &&
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	759 hdr->major_version != MAIL_INDEX_MAJOR_VERSION) {
2822 280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	760 /* major version change - handle silently */
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	761 return 0;
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	762 }
280c933983a9 Handle major version changes silently even if the file isn't large enough to Timo Sirainen <tss@iki.fi> parents: 2821 diff changeset	763
5058 2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	764 if (fstat(index->fd, &st) < 0) {
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	765 mail_index_set_syscall_error(index, "fstat()");
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	766 return -1;
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	767 }
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	768
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	769 if (ret >= 0 && pos >= MAIL_INDEX_HEADER_MIN_SIZE &&
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	770 (ret > 0 \|\| pos >= hdr->base_header_size)) {
5212 0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	771 if (!mail_index_check_header_compat(hdr)) {
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	772 /* Can't use this file */
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	773 return 0;
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	774 }
0866a2b60b2b Check index file's endianess earlier, otherwise we'll complain about other Timo Sirainen <tss@iki.fi> parents: 5060 diff changeset	775
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	776 if (hdr->base_header_size < MAIL_INDEX_HEADER_MIN_SIZE \|\|
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	777 hdr->header_size < hdr->base_header_size) {
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	778 mail_index_set_error(index, "Corrupted index file %s: "
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	779 "Corrupted header sizes (base %u, full %u)",
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	780 index->filepath, hdr->base_header_size,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	781 hdr->header_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	782 return 0;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	783 }
5059 baa0e9f60a24 cleanups Timo Sirainen <tss@iki.fi> parents: 5058 diff changeset	784 if (hdr->header_size > (uoff_t)st.st_size) {
5058 2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	785 mail_index_set_error(index, "Corrupted index file %s: "
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	786 "Corrupted header size (%u > %"PRIuUOFF_T")",
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	787 index->filepath, hdr->header_size,
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	788 st.st_size);
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	789 return 0;
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	790 }
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	791
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	792 if (pos > hdr->header_size)
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	793 pos = hdr->header_size;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	794
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	795 /* place the base header into memory. */
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	796 buffer_reset(map->hdr_copy_buf);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	797 buffer_append(map->hdr_copy_buf, buf, pos);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	798
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	799 if (pos != hdr->header_size) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	800 /* @UNSAFE: read the rest of the header into memory */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	801 data = buffer_append_space_unsafe(map->hdr_copy_buf,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	802 hdr->header_size -
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	803 pos);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	804 ret = pread_full(index->fd, data,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	805 hdr->header_size - pos, pos);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	806 }
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	807 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	808
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	809 if (ret > 0) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	810 /* header read, read the records now. */
5059 baa0e9f60a24 cleanups Timo Sirainen <tss@iki.fi> parents: 5058 diff changeset	811 records_size = (size_t)hdr->messages_count * hdr->record_size;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	812
5059 baa0e9f60a24 cleanups Timo Sirainen <tss@iki.fi> parents: 5058 diff changeset	813 if ((uoff_t)st.st_size - hdr->header_size < records_size \|\|
5060 0cc8f8fb18f3 Fix for last check Timo Sirainen <tss@iki.fi> parents: 5059 diff changeset	814 (hdr->record_size != 0 &&
0cc8f8fb18f3 Fix for last check Timo Sirainen <tss@iki.fi> parents: 5059 diff changeset	815 records_size / hdr->record_size != hdr->messages_count)) {
5058 2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	816 records_count = (st.st_size - hdr->header_size) /
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	817 hdr->record_size;
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	818 mail_index_set_error(index, "Corrupted index file %s: "
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	819 "messages_count too large (%u > %u)",
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	820 index->filepath, hdr->messages_count,
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	821 records_count);
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	822 return 0;
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	823 }
2a6b6575b421 mmap_disable=yes: Verify that header_size and messages_count are low enough Timo Sirainen <tss@iki.fi> parents: 5028 diff changeset	824
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	825 if (map->buffer == NULL) {
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	826 map->buffer = buffer_create_dynamic(default_pool,
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	827 records_size);
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	828 }
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	829
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	830 /* @UNSAFE */
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	831 buffer_set_used_size(map->buffer, 0);
8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	832 data = buffer_append_space_unsafe(map->buffer, records_size);
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	833
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	834 ret = pread_full(index->fd, data, records_size,
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	835 hdr->header_size);
1939 4698c121bae3 changes Timo Sirainen <tss@iki.fi> parents: 1935 diff changeset	836 }
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	837
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	838 if (ret < 0) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	839 if (errno == ESTALE && try_retry) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	840 /* a new index file was renamed over this one. */
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	841 *retry_r = TRUE;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	842 return 0;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	843 }
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	844 mail_index_set_syscall_error(index, "pread_full()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	845 return -1;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	846 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	847 if (ret == 0) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	848 mail_index_set_error(index,
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	849 "Corrupted index file %s: File too small",
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	850 index->filepath);
2824 431f677bc0b9 More rebuilding fixes with mmap_disable=yes Timo Sirainen <tss@iki.fi> parents: 2823 diff changeset	851 return 0;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	852 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	853
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	854 map->records = data;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	855 map->records_count = hdr->messages_count;
1930 8f31c6dae363 fixes Timo Sirainen <tss@iki.fi> parents: 1925 diff changeset	856
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	857 mail_index_map_copy_hdr(map, hdr);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	858 map->hdr_base = map->hdr_copy_buf->data;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	859
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	860 index->sync_log_file_seq = hdr->log_file_seq;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	861 index->sync_log_file_offset = hdr->log_file_int_offset;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	862 return 1;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	863 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	864
4237 20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	865 bool mail_index_is_ext_synced(struct mail_transaction_log_view *log_view,
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	866 struct mail_index_map *map)
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	867 {
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	868 uint32_t prev_seq;
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	869 uoff_t prev_offset;
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	870
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	871 mail_transaction_log_view_get_prev_pos(log_view, &prev_seq,
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	872 &prev_offset);
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	873 return prev_seq < map->hdr.log_file_seq \|\|
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	874 (prev_seq == map->hdr.log_file_seq &&
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	875 prev_offset < map->hdr.log_file_ext_offset);
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	876 }
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	877
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	878 static int mail_index_sync_from_transactions(struct mail_index *index,
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	879 struct mail_index_map **map,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	880 bool sync_to_index)
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	881 {
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	882 const struct mail_index_header map_hdr = &(map)->hdr;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	883 struct mail_index_view *view;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	884 struct mail_index_sync_map_ctx sync_map_ctx;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	885 struct mail_index_header hdr;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	886 const struct mail_transaction_header *thdr;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	887 const void *tdata;
2939 00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	888 uint32_t prev_seq, max_seq;
00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	889 uoff_t prev_offset, max_offset;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	890 size_t pos;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	891 int ret;
4278 19a0864deb4e If syncing from transaction log fails, fallback to re-reading the index file Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	892 bool skipped, check_ext_offsets, broken;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	893
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	894 if (sync_to_index) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	895 /* read the real log position where we are supposed to be
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	896 synced */
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	897 ret = mail_index_read_header(index, &hdr, sizeof(hdr), &pos);
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	898 if (ret < 0 && errno != ESTALE) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	899 mail_index_set_syscall_error(index, "pread()");
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	900 return -1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	901 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	902 if (pos < MAIL_INDEX_HEADER_MIN_SIZE)
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	903 return 0;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	904
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	905 if (map_hdr->log_file_seq == hdr.log_file_seq &&
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	906 map_hdr->log_file_int_offset == hdr.log_file_int_offset) {
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	907 /* nothing to do */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	908 return 1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	909 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	910
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	911 if (map_hdr->log_file_seq > hdr.log_file_seq \|\|
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	912 (map_hdr->log_file_seq == hdr.log_file_seq &&
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	913 map_hdr->log_file_int_offset > hdr.log_file_int_offset)) {
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	914 /* we went too far, have to re-read the file */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	915 return 0;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	916 }
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	917 if (map_hdr->log_file_ext_offset !=
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	918 map_hdr->log_file_int_offset \|\|
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	919 hdr.log_file_ext_offset != hdr.log_file_int_offset) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	920 /* too much trouble to get this right. */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	921 return 0;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	922 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	923 max_seq = hdr.log_file_seq;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	924 max_offset = hdr.log_file_int_offset;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	925 } else {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	926 /* sync everything there is */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	927 max_seq = (uint32_t)-1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	928 max_offset = (uoff_t)-1;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	929 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	930
5272 82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	931 index->map = *map;
82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	932
82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	933 view = mail_index_view_open(index);
82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	934 if (mail_transaction_log_view_set(view->log_view,
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	935 map_hdr->log_file_seq,
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	936 map_hdr->log_file_int_offset,
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	937 max_seq, max_offset,
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3831 diff changeset	938 MAIL_TRANSACTION_TYPE_MASK) <= 0) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3831 diff changeset	939 /* can't use it. sync by re-reading index. */
5272 82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	940 mail_index_view_close(&view);
5311 09b5de33144e Error handling crashfix Timo Sirainen <tss@iki.fi> parents: 5278 diff changeset	941 index->map = NULL;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	942 return 0;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	943 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	944
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	945 mail_index_sync_map_init(&sync_map_ctx, view,
4283 22b54c007b91 We were updating index->map also when syncing views. Timo Sirainen <tss@iki.fi> parents: 4278 diff changeset	946 MAIL_INDEX_SYNC_HANDLER_HEAD);
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	947
4278 19a0864deb4e If syncing from transaction log fails, fallback to re-reading the index file Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	948 check_ext_offsets = TRUE; broken = FALSE;
5272 82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	949 while ((ret = mail_transaction_log_view_next(view->log_view, &thdr,
82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	950 &tdata, &skipped)) > 0) {
4237 20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	951 if ((thdr->type & MAIL_TRANSACTION_EXTERNAL) != 0 &&
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	952 check_ext_offsets) {
5272 82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	953 if (mail_index_is_ext_synced(view->log_view,
82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	954 index->map))
4237 20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	955 continue;
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	956 check_ext_offsets = FALSE;
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	957 }
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	958
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	959 if (mail_index_sync_record(&sync_map_ctx, thdr, tdata) < 0) {
4278 19a0864deb4e If syncing from transaction log fails, fallback to re-reading the index file Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	960 ret = 0;
19a0864deb4e If syncing from transaction log fails, fallback to re-reading the index file Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	961 broken = TRUE;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	962 break;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	963 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	964 }
4278 19a0864deb4e If syncing from transaction log fails, fallback to re-reading the index file Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	965 if (ret == 0 && !broken)
19a0864deb4e If syncing from transaction log fails, fallback to re-reading the index file Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	966 ret = 1;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	967
5272 82b4a055e109 mmap_disable=yes crashes with recent changes. Timo Sirainen <tss@iki.fi> parents: 5247 diff changeset	968 mail_transaction_log_view_get_prev_pos(view->log_view, &prev_seq,
2939 00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	969 &prev_offset);
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3147 diff changeset	970 i_assert(prev_seq <= max_seq &&
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3147 diff changeset	971 (prev_seq != max_seq \|\| prev_offset <= max_offset));
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3147 diff changeset	972
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3147 diff changeset	973 index->map->hdr.log_file_seq = prev_seq;
2939 00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	974 index->map->hdr.log_file_int_offset =
00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	975 index->map->hdr.log_file_ext_offset = prev_offset;
00b78082cd50 mmap_disable=yes: when we're syncing index by reading transaction log, Timo Sirainen <tss@iki.fi> parents: 2938 diff changeset	976
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	977 mail_index_sync_map_deinit(&sync_map_ctx);
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	978 mail_index_view_close(&view);
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	979
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	980 *map = index->map;
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	981 index->map = NULL;
cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	982
4278 19a0864deb4e If syncing from transaction log fails, fallback to re-reading the index file Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	983 if (sync_to_index && ret > 0) {
3369 b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	984 /* make sure we did everything right. note that although the
b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	985 message counts should be equal, the flag counters may not */
2956 cd257e96a370 Added assert Timo Sirainen <tss@iki.fi> parents: 2940 diff changeset	986 i_assert(hdr.messages_count == (*map)->hdr.messages_count);
3369 b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	987 i_assert(hdr.log_file_seq == (*map)->hdr.log_file_seq);
b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	988 i_assert(hdr.log_file_int_offset == (*map)->hdr.log_file_int_offset);
b4a462cc8737 Fix to mmap_disable=yes syncing Timo Sirainen <tss@iki.fi> parents: 3353 diff changeset	989 i_assert(hdr.log_file_ext_offset == (*map)->hdr.log_file_ext_offset);
2896 305476a6f8ed When we actually read index file's header make sure it gets copied to memory Timo Sirainen <tss@iki.fi> parents: 2895 diff changeset	990 }
305476a6f8ed When we actually read index file's header make sure it gets copied to memory Timo Sirainen <tss@iki.fi> parents: 2895 diff changeset	991
4278 19a0864deb4e If syncing from transaction log fails, fallback to re-reading the index file Timo Sirainen <tss@iki.fi> parents: 4237 diff changeset	992 return ret;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	993 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	994
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	995 static int mail_index_read_map_with_retry(struct mail_index *index,
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	996 struct mail_index_map **map,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	997 bool sync_to_index)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	998 {
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	999 mail_index_sync_lost_handler_t const handlers;
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1000 unsigned int i, count;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1001 int ret;
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1002 bool retry;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1003
2938 bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	1004 if (index->log_locked) {
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	1005 /* we're most likely syncing the index and we really don't
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	1006 want to read more than what was synced last time. */
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	1007 sync_to_index = TRUE;
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	1008 }
bc8ff03715b2 mmap_disable=yes: if mail_index_map() is called while syncing make sure we Timo Sirainen <tss@iki.fi> parents: 2937 diff changeset	1009
3177 95d2d8d5958b Some error handling path crashfixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1010 if ((*map)->hdr.indexid != 0 && index->log != NULL) {
95d2d8d5958b Some error handling path crashfixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1011 /* we're not creating the index, or opening transaction log.
95d2d8d5958b Some error handling path crashfixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1012 sync this as a view from transaction log. */
2935 0b72e95cd3ae Don't try to fsck when we're modifying index. It could have happened with Timo Sirainen <tss@iki.fi> parents: 2934 diff changeset	1013 index->sync_update = TRUE;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1014 ret = mail_index_sync_from_transactions(index, map,
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1015 sync_to_index);
2935 0b72e95cd3ae Don't try to fsck when we're modifying index. It could have happened with Timo Sirainen <tss@iki.fi> parents: 2934 diff changeset	1016 index->sync_update = FALSE;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1017 if (ret != 0)
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1018 return ret;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1019
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1020 /* transaction log lost/broken, fallback to re-reading it */
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1021 }
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1022
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	1023 /* notify all "sync lost" handlers */
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1024 handlers = array_get(&index->sync_lost_handlers, &count);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1025 for (i = 0; i < count; i++)
2934 c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	1026 (*handlers[i])(index);
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index Timo Sirainen <tss@iki.fi> parents: 2926 diff changeset	1027
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1028 for (i = 0;; i++) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1029 ret = mail_index_read_map(index, *map, &retry,
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1030 i < MAIL_INDEX_ESTALE_RETRY_COUNT);
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1031 if (ret != 0 \|\| !retry)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1032 return ret;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1033
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1034 /* ESTALE - reopen index file */
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1035 if (close(index->fd) < 0)
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1036 mail_index_set_syscall_error(index, "close()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1037 index->fd = -1;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1038
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1039 ret = mail_index_try_open_only(index);
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1040 if (ret <= 0) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1041 if (ret == 0) {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1042 /* the file was lost */
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1043 errno = ENOENT;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1044 mail_index_set_syscall_error(index, "open()");
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1045 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1046 return -1;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1047 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1048 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1049 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1050
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1051 static int mail_index_map_try_existing(struct mail_index *index)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1052 {
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1053 struct mail_index_map *map = index->map;
1946 dbec6c9dbffa syncing fix Timo Sirainen <tss@iki.fi> parents: 1945 diff changeset	1054 const struct mail_index_header *hdr;
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1055 size_t used_size;
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1056 int ret;
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1057
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1058 if (MAIL_INDEX_MAP_IS_IN_MEMORY(map))
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1059 return 0;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1060
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1061 hdr = map->mmap_base;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1062
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1063 /* always check corrupted-flag to avoid errors later */
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1064 if ((hdr->flags & MAIL_INDEX_HDR_FLAG_CORRUPTED) != 0)
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1065 return -1;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1066
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1067 used_size = hdr->header_size + hdr->messages_count * hdr->record_size;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1068 if (map->mmap_size >= used_size && map->hdr_base == hdr) {
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1069 map->records_count = hdr->messages_count;
2922 fadf33c24902 With mmap_disabled=no the index header wasn't always updated causing various Timo Sirainen <tss@iki.fi> parents: 2921 diff changeset	1070 mail_index_map_copy_hdr(map, hdr);
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1071
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1072 /* make sure the header is still valid. it also re-parses
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1073 extensions although they shouldn't change without the whole
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1074 index being recreated */
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1075 ret = mail_index_check_header(index, map);
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1076 if (ret > 0)
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1077 return 1;
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1078 /* broken. fallback to re-mmaping which will catch it */
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1079 }
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1080 return 0;
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1081 }
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1082
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1083 int mail_index_map(struct mail_index *index, bool force)
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1084 {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1085 struct mail_index_map *map;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1086 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1087
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1088 i_assert(!index->mapping);
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	1089 i_assert(index->map == NULL \|\| index->map->refcount > 0);
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1090 i_assert(index->lock_type != F_UNLCK);
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1091
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1092 if (MAIL_INDEX_IS_IN_MEMORY(index)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1093 if (index->map == NULL)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1094 mail_index_create_in_memory(index, NULL);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1095 return 1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1096 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1097
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1098 index->mapping = TRUE;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1099
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1100 if (!force && index->map != NULL) {
2937 43deb226d769 Added more asserts. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1101 i_assert(index->hdr != NULL);
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1102 ret = mail_index_map_try_existing(index);
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1103 if (ret != 0) {
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1104 index->mapping = FALSE;
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1105 return ret;
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1106 }
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	1107
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	1108 if (index->lock_type == F_WRLCK) {
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	1109 /* we're syncing, don't break the mapping */
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1110 index->mapping = FALSE;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	1111 return 1;
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2825 diff changeset	1112 }
2553 1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1113 }
1fed92b6cc01 Avoid re-mmaping if not needed. Timo Sirainen <tss@iki.fi> parents: 2542 diff changeset	1114
2542 c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	1115 if (index->map != NULL && index->map->refcount > 1) {
c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	1116 /* this map is already used by some views and they may have
c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	1117 pointers into it. leave them and create a new mapping. */
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1118 if (!index->mmap_disable) {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1119 map = NULL;
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1120 } else {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1121 /* create a copy of the mapping instead so we don't
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1122 have to re-read it */
2911 f2c2b17d7659 Changed mail_index_map_to_memory() to mail_index_map_clone(). Even if the Timo Sirainen <tss@iki.fi> parents: 2902 diff changeset	1123 map = mail_index_map_clone(index->map,
f2c2b17d7659 Changed mail_index_map_to_memory() to mail_index_map_clone(). Even if the Timo Sirainen <tss@iki.fi> parents: 2902 diff changeset	1124 index->map->hdr.record_size);
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1125 }
2542 c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	1126 index->map->refcount--;
c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	1127 index->map = NULL;
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1128 } else {
1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1129 map = index->map;
2542 c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	1130 }
c3608081d0e4 When updating index mappings, don't let it affect existing views mappings. Timo Sirainen <tss@iki.fi> parents: 2409 diff changeset	1131
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	1132 if (map == NULL) {
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	1133 map = i_new(struct mail_index_map, 1);
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	1134 map->refcount = 1;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1135 map->hdr_copy_buf =
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1136 buffer_create_dynamic(default_pool, sizeof(map->hdr));
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	1137 } else if (MAIL_INDEX_MAP_IS_IN_MEMORY(map)) {
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1138 i_assert(!map->write_to_disk);
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2043 diff changeset	1139 } else if (map->mmap_base != NULL) {
2193 1019b7694a11 assertfix Timo Sirainen <tss@iki.fi> parents: 2170 diff changeset	1140 i_assert(map->buffer == NULL);
1946 dbec6c9dbffa syncing fix Timo Sirainen <tss@iki.fi> parents: 1945 diff changeset	1141 if (munmap(map->mmap_base, map->mmap_size) < 0)
dbec6c9dbffa syncing fix Timo Sirainen <tss@iki.fi> parents: 1945 diff changeset	1142 mail_index_set_syscall_error(index, "munmap()");
dbec6c9dbffa syncing fix Timo Sirainen <tss@iki.fi> parents: 1945 diff changeset	1143 map->mmap_base = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1144 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1145
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1146 index->hdr = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1147 index->map = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1148
2823 43bab10e060d Rebuild broken index files with mmap_disabled=yes too.. Timo Sirainen <tss@iki.fi> parents: 2822 diff changeset	1149 if (!index->mmap_disable)
43bab10e060d Rebuild broken index files with mmap_disabled=yes too.. Timo Sirainen <tss@iki.fi> parents: 2822 diff changeset	1150 ret = mail_index_mmap(index, map);
2885 1880f191278a Avoid re-reading index file with mmap_disabled=yes. Instead sync it by Timo Sirainen <tss@iki.fi> parents: 2884 diff changeset	1151 else
2913 cd293e5e6a71 mmap_disable=yes fixes. Timo Sirainen <tss@iki.fi> parents: 2911 diff changeset	1152 ret = mail_index_read_map_with_retry(index, &map, force);
2937 43deb226d769 Added more asserts. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1153 i_assert(index->map == NULL);
43deb226d769 Added more asserts. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1154
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1155 if (ret > 0) {
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1156 ret = mail_index_check_header(index, map);
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1157 if (ret < 0)
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1158 ret = 0;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1159 else if (ret == 0) {
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1160 index->fsck = TRUE;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1161 ret = 1;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1162 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1163 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1164
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1165 if (ret <= 0) {
3831 1a4d3c25feb8 Minor cleanups Timo Sirainen <tss@iki.fi> parents: 3824 diff changeset	1166 mail_index_map_clear(index, map);
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1167 mail_index_unmap(index, &map);
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1168 index->mapping = FALSE;
c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1169 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1170 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1171
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1172 index->hdr = &map->hdr;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1173 index->map = map;
2937 43deb226d769 Added more asserts. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1174 i_assert(map->hdr.messages_count == map->records_count);
3179 c620532e7bc8 Assert if mail_index_map() is called recursively. Timo Sirainen <tss@iki.fi> parents: 3177 diff changeset	1175 index->mapping = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1176 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1177 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1178
3296 c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1179 int mail_index_get_latest_header(struct mail_index *index,
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1180 struct mail_index_header *hdr_r)
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1181 {
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1182 size_t pos;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1183 unsigned int i;
3296 c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1184 int ret;
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1185
5273 9cdfd66cf5cc Rotating in-memory transaction log crashed Timo Sirainen <tss@iki.fi> parents: 5272 diff changeset	1186 if (MAIL_INDEX_IS_IN_MEMORY(index)) {
9cdfd66cf5cc Rotating in-memory transaction log crashed Timo Sirainen <tss@iki.fi> parents: 5272 diff changeset	1187 hdr_r = index->hdr;
9cdfd66cf5cc Rotating in-memory transaction log crashed Timo Sirainen <tss@iki.fi> parents: 5272 diff changeset	1188 return TRUE;
9cdfd66cf5cc Rotating in-memory transaction log crashed Timo Sirainen <tss@iki.fi> parents: 5272 diff changeset	1189 }
9cdfd66cf5cc Rotating in-memory transaction log crashed Timo Sirainen <tss@iki.fi> parents: 5272 diff changeset	1190
3296 c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1191 if (!index->mmap_disable) {
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1192 ret = mail_index_map(index, FALSE);
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1193 if (ret > 0)
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1194 hdr_r = index->hdr;
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1195 else
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1196 memset(hdr_r, 0, sizeof(*hdr_r));
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1197 return ret;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1198 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1199
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1200 for (i = 0;; i++) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1201 ret = mail_index_read_header(index, hdr_r, sizeof(*hdr_r),
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1202 &pos);
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1203 if (ret <= 0 \|\| errno != ESTALE \|\|
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1204 i == MAIL_INDEX_ESTALE_RETRY_COUNT)
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1205 break;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1206
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1207 /* ESTALE - reopen index file */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1208 if (close(index->fd) < 0)
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1209 mail_index_set_syscall_error(index, "close()");
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1210 index->fd = -1;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1211
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1212 ret = mail_index_try_open_only(index);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1213 if (ret <= 0) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1214 if (ret == 0) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1215 /* the file was lost */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1216 errno = ENOENT;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1217 mail_index_set_syscall_error(index, "open()");
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1218 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1219 return -1;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1220 }
3296 c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1221 }
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1222
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1223 if (ret < 0)
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1224 mail_index_set_syscall_error(index, "pread_full()");
3296 c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1225 return ret;
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1226 }
c2b8904dc21b mmap_disable=yes: Transaction log might have been rotated before everything Timo Sirainen <tss@iki.fi> parents: 3294 diff changeset	1227
2114 4aedcb4c51cb Added support for per-index sized mail_index_record. Timo Sirainen <tss@iki.fi> parents: 2095 diff changeset	1228 struct mail_index_map *
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1229 mail_index_map_clone(const struct mail_index_map *map, uint32_t new_record_size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1230 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1231 struct mail_index_map *mem_map;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1232 struct mail_index_header *hdr;
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1233 struct mail_index_ext *extensions;
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1234 void src, dest;
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1235 size_t size, copy_size;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1236 unsigned int i, count;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1237
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1238 size = map->records_count * new_record_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1239
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1240 mem_map = i_new(struct mail_index_map, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1241 mem_map->refcount = 1;
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1242 mem_map->buffer = buffer_create_dynamic(default_pool, size);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1243 if (map->hdr.record_size == new_record_size)
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1244 buffer_append(mem_map->buffer, map->records, size);
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1245 else {
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1246 copy_size = I_MIN(map->hdr.record_size, new_record_size);
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1247 src = map->records;
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1248 for (i = 0; i < map->records_count; i++) {
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1249 dest = buffer_append_space_unsafe(mem_map->buffer,
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1250 new_record_size);
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1251 memcpy(dest, src, copy_size);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1252 src = PTR_OFFSET(src, map->hdr.record_size);
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1253 }
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1254 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1255
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1256 mem_map->records = buffer_get_modifiable_data(mem_map->buffer, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1257 mem_map->records_count = map->records_count;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1258
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1259 mem_map->hdr_copy_buf =
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1260 buffer_create_dynamic(default_pool, map->hdr.header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1261 if (map->hdr.base_header_size < sizeof(*hdr))
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1262 buffer_append_zero(mem_map->hdr_copy_buf, sizeof(*hdr));
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1263 buffer_write(mem_map->hdr_copy_buf, 0,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1264 &map->hdr, map->hdr.base_header_size);
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1265 buffer_append(mem_map->hdr_copy_buf,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1266 CONST_PTR_OFFSET(map->hdr_base,
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1267 map->hdr.base_header_size),
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1268 map->hdr.header_size - map->hdr.base_header_size);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1269
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1270 hdr = buffer_get_modifiable_data(mem_map->hdr_copy_buf, NULL);
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1271 if (hdr->base_header_size < sizeof(*hdr))
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1272 hdr->base_header_size = sizeof(*hdr);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1273 hdr->record_size = new_record_size;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1274 mem_map->hdr = *hdr;
173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1275 mem_map->hdr_base = hdr;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1276
4312 2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1277 /* if we're syncing transaction log into memory and later use the
2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1278 mapping for updating the index, we need to remember what has
2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1279 changed */
2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1280 mem_map->write_atomic = map->write_atomic;
2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1281 if (map->write_to_disk) {
2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1282 mem_map->write_seq_first = map->write_seq_first;
2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1283 mem_map->write_seq_last = map->write_seq_last;
2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1284 }
2cb75f2e25c6 Keep track of write_atomic and write_seq_first/last while copying memory Timo Sirainen <tss@iki.fi> parents: 4283 diff changeset	1285
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1286 /* copy extensions */
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1287 if (array_is_created(&map->ext_id_map)) {
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1288 count = array_count(&map->ext_id_map);
2683 d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	1289 mail_index_map_init_extbufs(mem_map, count + 2);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1290
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1291 array_append_array(&mem_map->extensions, &map->extensions);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1292 array_append_array(&mem_map->ext_id_map, &map->ext_id_map);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1293
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1294 /* fix the name pointers to use our own pool */
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1295 extensions = array_get_modifiable(&mem_map->extensions, &count);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1296 for (i = 0; i < count; i++) {
5008 5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1297 i_assert(extensions[i].record_offset +
5ea657b6d83c Added some extra checks/asserts Timo Sirainen <tss@iki.fi> parents: 4982 diff changeset	1298 extensions[i].record_size <= hdr->record_size);
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1299 extensions[i].name = p_strdup(mem_map->extension_pool,
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1300 extensions[i].name);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1301 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1302 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1303
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1304 return mem_map;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1305 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1306
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1307 int mail_index_map_get_ext_idx(struct mail_index_map *map,
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2660 diff changeset	1308 uint32_t ext_id, uint32_t *idx_r)
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1309 {
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1310 const uint32_t *id;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1311
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1312 if (!array_is_created(&map->ext_id_map) \|\|
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1313 ext_id >= array_count(&map->ext_id_map))
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1314 return 0;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1315
3191 0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1316 id = array_idx(&map->ext_id_map, ext_id);
0df3f5d71958 Changed many buffers to arrays. Cleans up the code a lot. Timo Sirainen <tss@iki.fi> parents: 3179 diff changeset	1317 idx_r = id;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1318 return *idx_r != (uint32_t)-1;
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1319 }
5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1320
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1321 static int mail_index_try_open_only(struct mail_index *index)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1322 {
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1323 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1324
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1325 /* Note that our caller must close index->fd by itself.
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1326 mail_index_reopen() for example wants to revert back to old
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1327 index file if opening the new one fails. */
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4067 diff changeset	1328 index->fd = nfs_safe_open(index->filepath, O_RDWR);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1329 index->readonly = FALSE;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1330
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1331 if (index->fd == -1 && errno == EACCES) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1332 index->fd = open(index->filepath, O_RDONLY);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1333 index->readonly = TRUE;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1334 }
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1335
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1336 if (index->fd == -1) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1337 if (errno != ENOENT)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1338 return mail_index_set_syscall_error(index, "open()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1339
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1340 /* have to create it */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1341 return 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1342 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1343 return 1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1344 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1345
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1346 static int
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1347 mail_index_try_open(struct mail_index index, unsigned int lock_id_r)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1348 {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1349 unsigned int lock_id;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1350 int ret;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1351
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1352 i_assert(index->fd == -1);
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1353 i_assert(index->lock_type == F_UNLCK);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1354
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1355 if (lock_id_r != NULL)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1356 *lock_id_r = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1357
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1358 if (MAIL_INDEX_IS_IN_MEMORY(index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1359 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1360
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1361 ret = mail_index_try_open_only(index);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1362 if (ret <= 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1363 return ret;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1364
2409 9183098d7fdf Fixed fd leaks Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	1365 if (mail_index_lock_shared(index, FALSE, &lock_id) < 0) {
9183098d7fdf Fixed fd leaks Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	1366 (void)close(index->fd);
9183098d7fdf Fixed fd leaks Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	1367 index->fd = -1;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1368 return -1;
2409 9183098d7fdf Fixed fd leaks Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	1369 }
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1370 ret = mail_index_map(index, FALSE);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1371 if (ret == 0) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1372 /* it's corrupted - recreate it */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1373 mail_index_unlock(index, lock_id);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1374 if (lock_id_r != NULL)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1375 *lock_id_r = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1376
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1377 i_assert(index->file_lock == NULL);
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1378 (void)close(index->fd);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1379 index->fd = -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1380 } else {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1381 if (lock_id_r != NULL)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1382 *lock_id_r = lock_id;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1383 else
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1384 mail_index_unlock(index, lock_id);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1385 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1386 return ret;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1387 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1388
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1389 int mail_index_write_base_header(struct mail_index *index,
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1390 const struct mail_index_header *hdr)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1391 {
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1392 size_t hdr_size;
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1393
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1394 hdr_size = I_MIN(sizeof(*hdr), hdr->base_header_size);
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1395
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1396 if (!MAIL_INDEX_MAP_IS_IN_MEMORY(index->map)) {
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1397 memcpy(index->map->mmap_base, hdr, hdr_size);
51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1398 if (msync(index->map->mmap_base, hdr_size, MS_SYNC) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1399 return mail_index_set_syscall_error(index, "msync()");
3222 e38e04c7c67c mail_index_write_base_header() didn't update the header in memory with Timo Sirainen <tss@iki.fi> parents: 3191 diff changeset	1400 index->map->hdr = *hdr;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1401 } else {
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1402 if (!MAIL_INDEX_IS_IN_MEMORY(index)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1403 if (pwrite_full(index->fd, hdr, hdr_size, 0) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1404 mail_index_set_syscall_error(index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1405 "pwrite_full()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1406 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1407 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1408 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1409
2902 1022caefca74 mail_index_write_base_header() doesn't crash anymore if index mapping is in Timo Sirainen <tss@iki.fi> parents: 2896 diff changeset	1410 index->map->hdr = *hdr;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1411 buffer_write(index->map->hdr_copy_buf, 0, hdr, hdr_size);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1412 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1413
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1414 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1415 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1416
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1417 int mail_index_create_tmp_file(struct mail_index index, const char *path_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1418 {
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1419 mode_t old_mask;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1420 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1421 int fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1422
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1423 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1424
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1425 path = *path_r = t_strconcat(index->filepath, ".tmp", NULL);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1426 old_mask = umask(0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1427 fd = open(path, O_RDWR\|O_CREAT\|O_TRUNC, index->mode);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1428 umask(old_mask);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1429 if (fd == -1)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1430 return mail_index_file_set_syscall_error(index, path, "open()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1431
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1432 if (index->gid != (gid_t)-1 && fchown(fd, (uid_t)-1, index->gid) < 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1433 mail_index_file_set_syscall_error(index, path, "fchown()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1434 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1435 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1436
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1437 return fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1438 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1439
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1440 static int mail_index_create(struct mail_index *index,
2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1441 struct mail_index_header *hdr)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1442 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1443 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1444 uint32_t seq;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1445 uoff_t offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1446 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1447
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1448 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1449 i_assert(index->lock_type == F_UNLCK);
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1450
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1451 /* log file lock protects index creation */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1452 if (mail_transaction_log_sync_lock(index->log, &seq, &offset) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1453 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1454
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1455 ret = mail_index_try_open(index, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1456 if (ret != 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1457 mail_transaction_log_sync_unlock(index->log);
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1458 return ret < 0 ? -1 : 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1459 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1460
3582 bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1461 /* mark the existing log file as synced */
bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1462 hdr->log_file_seq = seq;
bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1463 hdr->log_file_int_offset = offset;
bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1464 hdr->log_file_ext_offset = offset;
bfa542369fdc When creating a new index file, mark its log offsets so that the possibly Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1465
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1466 /* create it fully in index.tmp first */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1467 index->fd = mail_index_create_tmp_file(index, &path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1468 if (index->fd == -1)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1469 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1470 else if (write_full(index->fd, hdr, sizeof(*hdr)) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1471 mail_index_file_set_syscall_error(index, path, "write_full()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1472 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1473 } else {
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1474 index->lock_type = F_WRLCK;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1475 ret = mail_index_map(index, FALSE);
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1476 index->lock_type = F_UNLCK;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1477 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1478
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1479 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1480 /* it's corrupted even while we just created it,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1481 should never happen unless someone pokes the file directly */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1482 mail_index_set_error(index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1483 "Newly created index file is corrupted: %s", path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1484 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1485 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1486
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1487 if (ret < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1488 if (unlink(path) < 0 && errno != ENOENT) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1489 mail_index_file_set_syscall_error(index, path,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1490 "unlink()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1491 }
2170 fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1492 } else {
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1493 /* make it visible to others */
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1494 if (rename(path, index->filepath) < 0) {
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1495 mail_index_set_error(index, "rename(%s, %s) failed: %m",
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1496 path, index->filepath);
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1497 ret = -1;
fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1498 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1499 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1500
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1501 mail_transaction_log_sync_unlock(index->log);
2170 fb785d235c43 minor fix for error handling Timo Sirainen <tss@iki.fi> parents: 2122 diff changeset	1502 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1503 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1504
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1505 static void mail_index_header_init(struct mail_index_header *hdr)
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1506 {
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1507 time_t now = time(NULL);
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1508
3105 f58ff8ec50c4 Added asserts Timo Sirainen <tss@iki.fi> parents: 3086 diff changeset	1509 i_assert((sizeof(*hdr) % sizeof(uint64_t)) == 0);
f58ff8ec50c4 Added asserts Timo Sirainen <tss@iki.fi> parents: 3086 diff changeset	1510
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1511 memset(hdr, 0, sizeof(*hdr));
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1512
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1513 hdr->major_version = MAIL_INDEX_MAJOR_VERSION;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1514 hdr->minor_version = MAIL_INDEX_MINOR_VERSION;
2116 fbb4da13ee3b Fixes for extra_records Timo Sirainen <tss@iki.fi> parents: 2115 diff changeset	1515 hdr->base_header_size = sizeof(*hdr);
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1516 hdr->header_size = sizeof(*hdr);
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1517 hdr->record_size = sizeof(struct mail_index_record);
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1518
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1519 #ifndef WORDS_BIGENDIAN
3819 61f0e1fe9b55 Moved uoff_t and time_t compatibility checks to cache file, since they're Timo Sirainen <tss@iki.fi> parents: 3818 diff changeset	1520 hdr->compat_flags \|= MAIL_INDEX_COMPAT_LITTLE_ENDIAN;
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1521 #endif
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1522
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1523 hdr->indexid = now;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1524
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1525 hdr->next_uid = 1;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1526 }
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2063 diff changeset	1527
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1528 static void mail_index_create_in_memory(struct mail_index *index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1529 const struct mail_index_header *hdr)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1530 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1531 struct mail_index_header tmp_hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1532 struct mail_index_map tmp_map;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1533
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1534 if (hdr == NULL) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1535 mail_index_header_init(&tmp_hdr);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1536 hdr = &tmp_hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1537 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1538
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1539 memset(&tmp_map, 0, sizeof(tmp_map));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1540 tmp_map.hdr = *hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1541 tmp_map.hdr_base = hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1542
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1543 /* a bit kludgy way to do this, but it initializes everything
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1544 nicely and correctly */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1545 index->map = mail_index_map_clone(&tmp_map, hdr->record_size);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1546 index->hdr = &index->map->hdr;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1547 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1548
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1549 /* returns -1 = error, 0 = won't create, 1 = ok */
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1550 static int mail_index_open_files(struct mail_index *index,
2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1551 enum mail_index_open_flags flags)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1552 {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1553 struct mail_index_header hdr;
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1930 diff changeset	1554 unsigned int lock_id = 0;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1555 int ret;
5247 c1ef15c1a1fa If index file is corrupted, recreate transaction log file instead of trying Timo Sirainen <tss@iki.fi> parents: 5224 diff changeset	1556 bool create = FALSE, created = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1557
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1930 diff changeset	1558 ret = mail_index_try_open(index, &lock_id);
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1559 if (ret > 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1560 hdr = *index->hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1561 else if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1562 /* doesn't exist, or corrupted */
3912 fc0b638330a4 Added mbox_min_index_size setting. Timo Sirainen <tss@iki.fi> parents: 3911 diff changeset	1563 if ((flags & MAIL_INDEX_OPEN_FLAG_CREATE) == 0 &&
fc0b638330a4 Added mbox_min_index_size setting. Timo Sirainen <tss@iki.fi> parents: 3911 diff changeset	1564 !MAIL_INDEX_IS_IN_MEMORY(index))
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1565 return 0;
2563 5752e5ac88f0 Save extra record/header infos into index file permanently. Timo Sirainen <tss@iki.fi> parents: 2553 diff changeset	1566 mail_index_header_init(&hdr);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1567 index->hdr = &hdr;
5247 c1ef15c1a1fa If index file is corrupted, recreate transaction log file instead of trying Timo Sirainen <tss@iki.fi> parents: 5224 diff changeset	1568 create = TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1569 } else if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1570 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1571
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1572 index->indexid = hdr.indexid;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1573
5247 c1ef15c1a1fa If index file is corrupted, recreate transaction log file instead of trying Timo Sirainen <tss@iki.fi> parents: 5224 diff changeset	1574 index->log = create ?
c1ef15c1a1fa If index file is corrupted, recreate transaction log file instead of trying Timo Sirainen <tss@iki.fi> parents: 5224 diff changeset	1575 mail_transaction_log_create(index) :
c1ef15c1a1fa If index file is corrupted, recreate transaction log file instead of trying Timo Sirainen <tss@iki.fi> parents: 5224 diff changeset	1576 mail_transaction_log_open_or_create(index);
5017 987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1577 if (index->log == NULL) {
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1578 if (ret == 0)
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1579 index->hdr = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1580 return -1;
5017 987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1581 }
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1582
5278 d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5274 diff changeset	1583 if (index->map == NULL) {
5017 987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1584 mail_index_header_init(&hdr);
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1585 index->hdr = &hdr;
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1586
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1587 /* index->indexid may be updated by transaction log opening,
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1588 in case someone else had already created a new log file */
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1589 hdr.indexid = index->indexid;
3556 6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	1590
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1591 if (lock_id != 0) {
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1592 mail_index_unlock(index, lock_id);
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1593 lock_id = 0;
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1594 }
5017 987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1595
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1596 if (!MAIL_INDEX_IS_IN_MEMORY(index)) {
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1597 if (mail_index_create(index, &hdr) < 0) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1598 /* fallback to in-memory index */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1599 mail_index_move_to_memory(index);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1600 mail_index_create_in_memory(index, &hdr);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1601 }
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1602 } else {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1603 mail_index_create_in_memory(index, &hdr);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1604 }
3129 1ea614abead3 Don't complain about cache indexid changing when recreating index. Timo Sirainen <tss@iki.fi> parents: 3105 diff changeset	1605 created = TRUE;
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1606 }
5017 987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5011 diff changeset	1607 i_assert(index->hdr != &hdr);
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1608
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1609 if (lock_id == 0) {
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1610 if (mail_index_lock_shared(index, FALSE, &lock_id) < 0)
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1611 return -1;
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1612
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1613 }
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1614
3129 1ea614abead3 Don't complain about cache indexid changing when recreating index. Timo Sirainen <tss@iki.fi> parents: 3105 diff changeset	1615 index->cache = created ? mail_cache_create(index) :
1ea614abead3 Don't complain about cache indexid changing when recreating index. Timo Sirainen <tss@iki.fi> parents: 3105 diff changeset	1616 mail_cache_open_or_create(index);
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1617
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1618 mail_index_unlock(index, lock_id);
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1619 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1620 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1621
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2715 diff changeset	1622 int mail_index_open(struct mail_index *index, enum mail_index_open_flags flags,
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1623 enum file_lock_method lock_method)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1624 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1625 int i = 0, ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1626
2926 c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1627 if (index->opened) {
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1628 if (index->hdr != NULL &&
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1629 (index->hdr->flags & MAIL_INDEX_HDR_FLAG_CORRUPTED) != 0) {
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1630 /* corrupted, reopen files */
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1631 mail_index_close(index);
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1632 } else {
4067 96ae9c096bde mail_index_open(): If index is already open, return 1 instead of 0 so the Timo Sirainen <tss@iki.fi> parents: 4053 diff changeset	1633 return 1;
2926 c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1634 }
c035583b4839 Make sure index is reopened if it's marked corrupted. Timo Sirainen <tss@iki.fi> parents: 2922 diff changeset	1635 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1636
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1637 index->filepath = MAIL_INDEX_IS_IN_MEMORY(index) ?
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1638 i_strdup("(in-memory index)") :
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1639 i_strconcat(index->dir, "/", index->prefix, NULL);
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1965 diff changeset	1640
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3819 diff changeset	1641 for (;;) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1642 index->shared_lock_count = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1643 index->excl_lock_count = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1644 index->lock_type = F_UNLCK;
2021 19e638ade25c Locking fixes Timo Sirainen <tss@iki.fi> parents: 1984 diff changeset	1645 index->lock_id = 2;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1646
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2715 diff changeset	1647 index->readonly = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1648 index->nodiskspace = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1649 index->index_lock_timeout = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1650 index->log_locked = FALSE;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1651 index->mmap_disable =
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1652 (flags & MAIL_INDEX_OPEN_FLAG_MMAP_DISABLE) != 0;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1653 index->mmap_no_write =
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1654 (flags & MAIL_INDEX_OPEN_FLAG_MMAP_NO_WRITE) != 0;
4968 1baf9dd3fc40 Added dotlock_use_excl setting. Timo Sirainen <tss@iki.fi> parents: 4919 diff changeset	1655 index->use_excl_dotlocks =
1baf9dd3fc40 Added dotlock_use_excl setting. Timo Sirainen <tss@iki.fi> parents: 4919 diff changeset	1656 (flags & MAIL_INDEX_OPEN_FLAG_DOTLOCK_USE_EXCL) != 0;
4918 2f9173e103fd Added fsync_disable setting. Also added missing fsync()ing to dbox when Timo Sirainen <tss@iki.fi> parents: 4906 diff changeset	1657 index->fsync_disable =
2f9173e103fd Added fsync_disable setting. Also added missing fsync()ing to dbox when Timo Sirainen <tss@iki.fi> parents: 4906 diff changeset	1658 (flags & MAIL_INDEX_OPEN_FLAG_FSYNC_DISABLE) != 0;
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2715 diff changeset	1659 index->lock_method = lock_method;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1660
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1661 /* don't even bother to handle dotlocking without mmap being
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1662 disabled. that combination simply doesn't make any sense */
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1663 if (lock_method == FILE_LOCK_METHOD_DOTLOCK &&
2940 0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1664 !index->mmap_disable) {
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1665 i_fatal("lock_method=dotlock and mmap_disable=no "
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1666 "combination isn't supported. "
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1667 "You don't _really_ want it anyway.");
0e1481357fbb Changed assert() to friendly human readable message. Timo Sirainen <tss@iki.fi> parents: 2939 diff changeset	1668 }
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1669
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1939 diff changeset	1670 ret = mail_index_open_files(index, flags);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1671 if (ret <= 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1672 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1673
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1674 index->opened = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1675 if (index->fsck) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1676 index->fsck = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1677 ret = mail_index_fsck(index);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1678 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1679 /* completely broken, reopen */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1680 if (i++ < 3)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1681 continue;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1682 /* too many tries */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1683 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1684 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1685 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1686 break;
3824 3e756ea0474b Comment updates, small cleanups and optimizations. Timo Sirainen <tss@iki.fi> parents: 3819 diff changeset	1687 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1688
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1689 if (ret <= 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1690 mail_index_close(index);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1691
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1692 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1693 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1694
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1695 void mail_index_close(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1696 {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1697 if (index->log != NULL)
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1698 mail_transaction_log_close(&index->log);
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1699 if (index->map != NULL)
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1700 mail_index_unmap(index, &index->map);
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1701 if (index->cache != NULL)
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1702 mail_cache_free(&index->cache);
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1703 if (index->file_lock != NULL)
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1704 file_lock_free(&index->file_lock);
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 2193 diff changeset	1705
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1706 if (index->fd != -1) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1707 if (close(index->fd) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1708 mail_index_set_syscall_error(index, "close()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1709 index->fd = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1710 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1711
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1712 i_free_and_null(index->copy_lock_path);
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1713 i_free_and_null(index->filepath);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1714
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1715 index->indexid = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1716 index->opened = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1717 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1718
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1719 int mail_index_reopen(struct mail_index *index, int fd)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1720 {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1721 struct mail_index_map *old_map;
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1722 struct file_lock *old_file_lock;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1723 unsigned int old_shared_locks, old_lock_id, lock_id = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1724 int ret, old_fd, old_lock_type;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1725
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1726 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1727 i_assert(index->excl_lock_count == 0);
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1728
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1729 old_map = index->map;
5323 5c8f84bc07b1 Error handling fixes Timo Sirainen <tss@iki.fi> parents: 5311 diff changeset	1730 if (old_map != NULL)
5c8f84bc07b1 Error handling fixes Timo Sirainen <tss@iki.fi> parents: 5311 diff changeset	1731 old_map->refcount++;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1732 old_fd = index->fd;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1733
3551 65dc45982df9 Removed fcntl() text from some locking error messages, because if Timo Sirainen <tss@iki.fi> parents: 3550 diff changeset	1734 /* new file, new locks. the old fd can keep its locks, they don't
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1735 matter anymore as no-one's going to modify the file. */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1736 old_lock_type = index->lock_type;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1737 old_lock_id = index->lock_id;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1738 old_shared_locks = index->shared_lock_count;
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1739 old_file_lock = index->file_lock;
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1740
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1741 if (index->lock_type == F_RDLCK)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1742 index->lock_type = F_UNLCK;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1743 index->lock_id += 2;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1744 index->shared_lock_count = 0;
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1745 index->file_lock = NULL;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1746
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1747 if (fd != -1) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1748 index->fd = fd;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1749 ret = 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1750 } else {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1751 ret = mail_index_try_open_only(index);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1752 if (ret > 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1753 ret = mail_index_lock_shared(index, FALSE, &lock_id);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1754 else if (ret == 0) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1755 /* index file is lost */
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1756 ret = -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1757 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1758 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1759
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1760 if (ret == 0) {
2895 51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1761 /* read the new mapping. note that with mmap_disable we want
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1762 to keep the old mapping in index->map so we can update it
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1763 by reading transaction log. */
51be21fc2c9d Avoid re-reading index file with mmap_disable=yes. Timo Sirainen <tss@iki.fi> parents: 2885 diff changeset	1764 if (mail_index_map(index, TRUE) <= 0)
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1765 ret = -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1766 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1767
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1768 if (lock_id != 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1769 mail_index_unlock(index, lock_id);
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1770
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1771 if (ret == 0) {
5323 5c8f84bc07b1 Error handling fixes Timo Sirainen <tss@iki.fi> parents: 5311 diff changeset	1772 if (old_map != NULL)
5c8f84bc07b1 Error handling fixes Timo Sirainen <tss@iki.fi> parents: 5311 diff changeset	1773 mail_index_unmap(index, &old_map);
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1774 if (old_file_lock != NULL)
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1775 file_lock_free(&old_file_lock);
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1776 if (close(old_fd) < 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1777 mail_index_set_syscall_error(index, "close()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1778 } else {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1779 if (index->map != NULL)
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1780 mail_index_unmap(index, &index->map);
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1781
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1782 if (index->fd != -1) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1783 if (close(index->fd) < 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1784 mail_index_set_syscall_error(index, "close()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1785 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1786
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1787 index->map = old_map;
2884 173a0296e81d Update view's header only after syncing the view. This and other changes fix Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	1788 index->hdr = &index->map->hdr;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1789 index->fd = old_fd;
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1790 index->file_lock = old_file_lock;
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1791 index->lock_type = old_lock_type;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1792 index->lock_id = old_lock_id;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1793 index->shared_lock_count = old_shared_locks;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1794 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1795 return ret;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1796 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1797
2936 57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1798 int mail_index_reopen_if_needed(struct mail_index *index)
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1799 {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1800 struct stat st1, st2;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1801
4026 20c335412d7d Added some locking related asserts, comments and when replacing index file, open it Timo Sirainen <timo.sirainen@movial.fi> parents: 4023 diff changeset	1802 i_assert(index->copy_lock_path == NULL);
20c335412d7d Added some locking related asserts, comments and when replacing index file, open it Timo Sirainen <timo.sirainen@movial.fi> parents: 4023 diff changeset	1803
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1804 if (MAIL_INDEX_IS_IN_MEMORY(index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1805 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1806
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1807 if (fstat(index->fd, &st1) < 0) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1808 if (errno == ESTALE) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1809 /* deleted, reopen */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1810 if (mail_index_reopen(index, -1) < 0)
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1811 return -1;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1812 return 1;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1813 }
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1814 return mail_index_set_syscall_error(index, "fstat()");
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3912 diff changeset	1815 }
4225 abdef9d86a94 Added nfs_safe_stat() to transparently work around ESTALE with stat(). Timo Sirainen <timo.sirainen@movial.fi> parents: 4220 diff changeset	1816 if (nfs_safe_stat(index->filepath, &st2) < 0) {
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1817 mail_index_set_syscall_error(index, "stat()");
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1818 if (errno != ENOENT)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1819 return -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1820
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4394 diff changeset	1821 /* lost it? recreate later */
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4394 diff changeset	1822 mail_index_mark_corrupted(index);
2063 21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1823 return -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1824 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1825
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1826 if (st1.st_ino != st2.st_ino \|\|
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1827 !CMP_DEV_T(st1.st_dev, st2.st_dev)) {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1828 if (mail_index_reopen(index, -1) < 0)
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1829 return -1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1830 return 1;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1831 } else {
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1832 return 0;
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1833 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1834 }
21a3e5d08290 mail_index_refresh() - allows forcing a refresh check Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1835
2936 57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1836 int mail_index_refresh(struct mail_index *index)
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1837 {
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1838 unsigned int lock_id;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1839 int ret;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1840
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1841 if (MAIL_INDEX_IS_IN_MEMORY(index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1842 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1843
2936 57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1844 if (index->excl_lock_count > 0) {
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1845 /* we have index exclusively locked, nothing could
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1846 have changed. */
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1847 return 0;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1848 }
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1849
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1850 if (!index->mmap_disable) {
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1851 /* reopening is all we need */
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1852 return mail_index_reopen_if_needed(index);
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1853 }
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1854
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	1855 i_assert(!index->mapping);
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	1856
2936 57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1857 /* mail_index_map() simply reads latest changes from transaction log,
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1858 which makes us fully refreshed. */
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1859 if (mail_index_lock_shared(index, TRUE, &lock_id) < 0)
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1860 return -1;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1861
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1862 ret = mail_index_map(index, FALSE);
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1863 mail_index_unlock(index, lock_id);
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1864 return ret <= 0 ? -1 : 0;
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1865 }
57f975a9b63b Renamed mail_index_refresh() to mail_index_reopen_if_needed(). Added public Timo Sirainen <tss@iki.fi> parents: 2935 diff changeset	1866
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1867 struct mail_cache mail_index_get_cache(struct mail_index index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1868 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1869 return index->cache;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1870 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1871
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1872 int mail_index_set_error(struct mail_index index, const char fmt, ...)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1873 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1874 va_list va;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1875
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1876 i_free(index->error);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1877
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1878 if (fmt == NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1879 index->error = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1880 else {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1881 va_start(va, fmt);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1882 index->error = i_strdup_vprintf(fmt, va);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1883 va_end(va);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1884
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1885 i_error("%s", index->error);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1886 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1887
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1888 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1889 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1890
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1891 void mail_index_set_inconsistent(struct mail_index *index)
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1892 {
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1893 index->indexid = 0;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1894 }
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1895
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1896 int mail_index_move_to_memory(struct mail_index *index)
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1897 {
3911 3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1898 struct mail_index_map *map;
3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1899 int ret = 0;
3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1900
4394 0462137fa4ce mail_index_move_to_memory(): Don't break if the index is already in memory. Timo Sirainen <tss@iki.fi> parents: 4312 diff changeset	1901 if (MAIL_INDEX_IS_IN_MEMORY(index))
5430 c8bb83980ff7 If mail_index_move_to_memory() fails, make sure the future calls to it also Timo Sirainen <tss@iki.fi> parents: 5323 diff changeset	1902 return index->map == NULL ? -1 : 0;
4394 0462137fa4ce mail_index_move_to_memory(): Don't break if the index is already in memory. Timo Sirainen <tss@iki.fi> parents: 4312 diff changeset	1903
4919 d54145736f99 If mail_index_move_to_memory() was called before index file was opened, we Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	1904 /* set the index as being into memory */
d54145736f99 If mail_index_move_to_memory() was called before index file was opened, we Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	1905 i_free_and_null(index->dir);
d54145736f99 If mail_index_move_to_memory() was called before index file was opened, we Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	1906
5274 7641e625fbd5 When moving index file to memory, change the filename to be "(in-memory Timo Sirainen <tss@iki.fi> parents: 5273 diff changeset	1907 i_free(index->filepath);
7641e625fbd5 When moving index file to memory, change the filename to be "(in-memory Timo Sirainen <tss@iki.fi> parents: 5273 diff changeset	1908 index->filepath = i_strdup("(in-memory index)");
7641e625fbd5 When moving index file to memory, change the filename to be "(in-memory Timo Sirainen <tss@iki.fi> parents: 5273 diff changeset	1909
3912 fc0b638330a4 Added mbox_min_index_size setting. Timo Sirainen <tss@iki.fi> parents: 3911 diff changeset	1910 if (index->map == NULL) {
5224 c3ab7e621fa6 comment cleanup Timo Sirainen <tss@iki.fi> parents: 5212 diff changeset	1911 /* index was never even opened. just mark it as being in
3912 fc0b638330a4 Added mbox_min_index_size setting. Timo Sirainen <tss@iki.fi> parents: 3911 diff changeset	1912 memory and let the caller re-open the index. */
fc0b638330a4 Added mbox_min_index_size setting. Timo Sirainen <tss@iki.fi> parents: 3911 diff changeset	1913 i_assert(index->fd == -1);
fc0b638330a4 Added mbox_min_index_size setting. Timo Sirainen <tss@iki.fi> parents: 3911 diff changeset	1914 return -1;
fc0b638330a4 Added mbox_min_index_size setting. Timo Sirainen <tss@iki.fi> parents: 3911 diff changeset	1915 }
fc0b638330a4 Added mbox_min_index_size setting. Timo Sirainen <tss@iki.fi> parents: 3911 diff changeset	1916
3911 3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1917 /* move index map to memory */
5278 d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5274 diff changeset	1918 if (!MAIL_INDEX_MAP_IS_IN_MEMORY(index->map)) {
d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5274 diff changeset	1919 map = mail_index_map_clone(index->map,
d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5274 diff changeset	1920 index->map->hdr.record_size);
d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5274 diff changeset	1921 mail_index_unmap(index, &index->map);
d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5274 diff changeset	1922 index->map = map;
d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5274 diff changeset	1923 index->hdr = &map->hdr;
d569caa88a66 If we run out of disk space, move to in-memory indexes. Timo Sirainen <tss@iki.fi> parents: 5274 diff changeset	1924 }
3911 3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1925
4053 94b3363165c8 If transaction log opening fails, don't crash when trying to move it to Timo Sirainen <tss@iki.fi> parents: 4026 diff changeset	1926 if (index->log != NULL) {
94b3363165c8 If transaction log opening fails, don't crash when trying to move it to Timo Sirainen <tss@iki.fi> parents: 4026 diff changeset	1927 /* move transaction log to memory */
94b3363165c8 If transaction log opening fails, don't crash when trying to move it to Timo Sirainen <tss@iki.fi> parents: 4026 diff changeset	1928 if (mail_transaction_log_move_to_memory(index->log) < 0)
94b3363165c8 If transaction log opening fails, don't crash when trying to move it to Timo Sirainen <tss@iki.fi> parents: 4026 diff changeset	1929 ret = -1;
94b3363165c8 If transaction log opening fails, don't crash when trying to move it to Timo Sirainen <tss@iki.fi> parents: 4026 diff changeset	1930 }
3911 3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1931
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1932 if (index->file_lock != NULL)
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1933 file_lock_free(&index->file_lock);
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4795 diff changeset	1934
3911 3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1935 /* close the index file. */
3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1936 if (close(index->fd) < 0)
3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1937 mail_index_set_syscall_error(index, "close()");
3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1938 index->fd = -1;
3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1939
3a2fe49912f3 Fixed mail_index_move_to_memory() to work better and made it public Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1940 return ret;
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1941 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1942
1945 268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1943 void mail_index_mark_corrupted(struct mail_index *index)
1943 e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1944 {
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1945 struct mail_index_header hdr;
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1946
1945 268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1947 mail_index_set_inconsistent(index);
268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1948
4662 d021f2b0ded2 Handle moving to memory failures better so we won't crash in some Timo Sirainen <tss@iki.fi> parents: 4616 diff changeset	1949 if (index->readonly \|\| index->map == NULL)
1945 268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1950 return;
1943 e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1951
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1952 hdr = *index->hdr;
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1953 hdr.flags \|= MAIL_INDEX_HDR_FLAG_CORRUPTED;
2240 51719889ffcf Record size is allowed to change between index files. This will allow adding Timo Sirainen <tss@iki.fi> parents: 2209 diff changeset	1954 if (mail_index_write_base_header(index, &hdr) == 0) {
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3231 diff changeset	1955 if (!MAIL_INDEX_IS_IN_MEMORY(index) && fsync(index->fd) < 0)
1945 268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1956 mail_index_set_syscall_error(index, "fsync()");
268a4096e8be cleanup Timo Sirainen <tss@iki.fi> parents: 1943 diff changeset	1957 }
1943 e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1958 }
e473d2df21be missing index file fixes Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	1959
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1960 int mail_index_set_syscall_error(struct mail_index *index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1961 const char *function)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1962 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1963 i_assert(function != NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1964
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1965 if (ENOSPACE(errno)) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1966 index->nodiskspace = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1967 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1968 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1969
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1970 return mail_index_set_error(index, "%s failed with index file %s: %m",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1971 function, index->filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1972 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1973
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1974 int mail_index_file_set_syscall_error(struct mail_index *index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1975 const char *filepath,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1976 const char *function)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1977 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1978 i_assert(filepath != NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1979 i_assert(function != NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1980
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1981 if (ENOSPACE(errno)) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1982 index->nodiskspace = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1983 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1984 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1985
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1986 return mail_index_set_error(index, "%s failed with file %s: %m",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1987 function, filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1988 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1989
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1990 enum mail_index_error mail_index_get_last_error(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1991 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1992 if (index->nodiskspace)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1993 return MAIL_INDEX_ERROR_DISKSPACE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1994 if (index->error != NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1995 return MAIL_INDEX_ERROR_INTERNAL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1996
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1997 return MAIL_INDEX_ERROR_NONE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1998 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1999
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2000 const char mail_index_get_error_message(struct mail_index index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2001 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2002 return index->error;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2003 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2004
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2005 void mail_index_reset_error(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2006 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2007 if (index->error != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2008 i_free(index->error);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2009 index->error = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2010 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2011
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2012 index->nodiskspace = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2013 index->index_lock_timeout = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2014 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2015
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2016 uint32_t mail_index_uint32_to_offset(uint32_t offset)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2017 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2018 unsigned char buf[4];
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2019
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2020 i_assert(offset < 0x40000000);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2021 i_assert((offset & 3) == 0);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2022
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2023 offset >>= 2;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2024 buf[0] = 0x80 \| ((offset & 0x0fe00000) >> 21);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2025 buf[1] = 0x80 \| ((offset & 0x001fc000) >> 14);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2026 buf[2] = 0x80 \| ((offset & 0x00003f80) >> 7);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2027 buf[3] = 0x80 \| (offset & 0x0000007f);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2028 return ((uint32_t ) buf);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2029 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2030
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2031 uint32_t mail_index_offset_to_uint32(uint32_t offset)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2032 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2033 const unsigned char buf = (const unsigned char ) &offset;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2034
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2035 if ((offset & 0x80808080) != 0x80808080)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2036 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2037
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2038 return (((uint32_t)buf[3] & 0x7f) << 2) \|
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2039 (((uint32_t)buf[2] & 0x7f) << 9) \|
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2040 (((uint32_t)buf[1] & 0x7f) << 16) \|
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2041 (((uint32_t)buf[0] & 0x7f) << 23);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2296 diff changeset	2042 }

Mercurial > dovecot > core-2.2

annotate src/lib-index/mail-index.c @ 5453:a0a1432bb4e8 HEAD