dovecot/original-hg/dovecot-1.2: src/lib-index/mail-cache-transaction.c annotate

annotate src/lib-index/mail-cache-transaction.c @ 7138:876c7bca351c HEAD

Link cache records together directly when writing the new records, instead of delaying them until later and causing lots of small writes. We still do this delayed check and do the writes when it's required, but it shouldn't happen normally.

author	Timo Sirainen <tss@iki.fi>
date	Thu, 10 Jan 2008 04:23:23 +0200
parents	f0ad529ac9ea
children	24266b9a6c55

rev	line source
7086 7ed926ed7aa4 Updated copyright notices to include year 2008. Timo Sirainen <tss@iki.fi> parents: 7054 diff changeset	1 /* Copyright (c) 2003-2008 Dovecot authors, see the included COPYING file */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
6296 205ee38f10d1 Drop fields that haven't been used for 30 days when compressing. Timo Sirainen <tss@iki.fi> parents: 6004 diff changeset	4 #include "ioloop.h"
3470 346a494c2feb Moved array declaration to array-decl.h and include it in lib.h. So array.h Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	5 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	6 #include "buffer.h"
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	7 #include "file-cache.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	8 #include "file-set-size.h"
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	9 #include "read-full.h"
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	10 #include "write-full.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	11 #include "mail-cache-private.h"
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	12 #include "mail-index-transaction-private.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	13
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2247 diff changeset	14 #include <stddef.h>
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	15 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	16
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	17 #define MAIL_CACHE_WRITE_BUFFER 32768
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	18
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	19 struct mail_cache_reservation {
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	20 uint32_t offset;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	21 uint32_t size;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	22 };
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	23
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	24 struct mail_cache_transaction_ctx {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	25 struct mail_cache *cache;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	26 struct mail_cache_view *view;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	27 struct mail_index_transaction *trans;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	28
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	29 uint32_t cache_file_seq;
6841 0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	30 uint32_t first_new_seq;
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	31
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	32 buffer_t *cache_data;
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4261 diff changeset	33 ARRAY_DEFINE(cache_data_seq, uint32_t);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	34 uint32_t prev_seq;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	35 size_t prev_pos;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	36
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4261 diff changeset	37 ARRAY_DEFINE(reservations, struct mail_cache_reservation);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	38 uint32_t reserved_space_offset, reserved_space;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	39 uint32_t last_grow_size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	40
5947 7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	41 unsigned int tried_compression:1;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	42 unsigned int changes:1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	43 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	44
2354 1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	45 static int mail_cache_link_unlocked(struct mail_cache *cache,
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	46 uint32_t old_offset, uint32_t new_offset);
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	47
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	48 struct mail_cache_transaction_ctx *
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	49 mail_cache_get_transaction(struct mail_cache_view *view,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	50 struct mail_index_transaction *t)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	51 {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	52 struct mail_cache_transaction_ctx *ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	53
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	54 if (t->cache_trans_ctx != NULL)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	55 return t->cache_trans_ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	56
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	57 ctx = i_new(struct mail_cache_transaction_ctx, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 ctx->cache = view->cache;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	59 ctx->view = view;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	60 ctx->trans = t;
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	61 i_array_init(&ctx->reservations, 32);
2597 18cddf71996a Don't crash when cache isn't usable. Timo Sirainen <tss@iki.fi> parents: 2596 diff changeset	62
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	63 i_assert(view->transaction == NULL);
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	64 view->transaction = ctx;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2747 diff changeset	65 view->trans_view = mail_index_transaction_open_updated_view(t);
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	66
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	67 t->cache_trans_ctx = ctx;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	68 return ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	69 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	70
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	71 static void mail_cache_transaction_reset(struct mail_cache_transaction_ctx *ctx)
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	72 {
5181 922672fcdfdc Crashfix in error conditions. Timo Sirainen <tss@iki.fi> parents: 5099 diff changeset	73 ctx->cache_file_seq = MAIL_CACHE_IS_UNUSABLE(ctx->cache) ? 0 :
922672fcdfdc Crashfix in error conditions. Timo Sirainen <tss@iki.fi> parents: 5099 diff changeset	74 ctx->cache->hdr->file_seq;
5947 7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	75 mail_index_ext_set_reset_id(ctx->trans, ctx->cache->ext_id,
7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	76 ctx->cache_file_seq);
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	77
6646 f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	78 if (ctx->cache_data != NULL)
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	79 buffer_set_used_size(ctx->cache_data, 0);
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	80 if (array_is_created(&ctx->cache_data_seq))
f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	81 array_clear(&ctx->cache_data_seq);
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	82 ctx->prev_seq = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	83 ctx->prev_pos = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	84
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	85 array_clear(&ctx->reservations);
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	86 ctx->reserved_space_offset = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	87 ctx->reserved_space = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	88 ctx->last_grow_size = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	89
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	90 ctx->changes = FALSE;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	91 }
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	92
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	93 static void
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	94 mail_cache_transaction_free(struct mail_cache_transaction_ctx **_ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	95 {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	96 struct mail_cache_transaction_ctx ctx = _ctx;
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	97
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	98 *_ctx = NULL;
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	99
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	100 ctx->trans->cache_trans_ctx = NULL;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	101 ctx->view->transaction = NULL;
2315 22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	102 ctx->view->trans_seq1 = ctx->view->trans_seq2 = 0;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	103
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	104 mail_index_view_close(&ctx->view->trans_view);
2324 ea745d0c9c4b Delay memory allocations for cache transaction. We don't always need it. Timo Sirainen <tss@iki.fi> parents: 2315 diff changeset	105 if (ctx->cache_data != NULL)
6414 a6a49d5efc59 Changed buffer_free() and buffer_free_without_data() APIs to take ** pointer Timo Sirainen <tss@iki.fi> parents: 6296 diff changeset	106 buffer_free(&ctx->cache_data);
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	107 if (array_is_created(&ctx->cache_data_seq))
f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	108 array_free(&ctx->cache_data_seq);
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	109 array_free(&ctx->reservations);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	110 i_free(ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	111 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	112
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	113 static int
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	114 mail_cache_transaction_compress(struct mail_cache_transaction_ctx *ctx)
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	115 {
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	116 struct mail_cache *cache = ctx->cache;
6831 ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	117 struct mail_index_view *view;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	118 struct mail_index_transaction *trans;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	119 uint32_t log_file_seq;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	120 uoff_t log_file_offset;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	121 int ret;
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	122
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	123 ctx->tried_compression = TRUE;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	124
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	125 cache->need_compress_file_seq =
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	126 MAIL_CACHE_IS_UNUSABLE(cache) ? 0 : cache->hdr->file_seq;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	127
6831 ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	128 view = mail_index_view_open(cache->index);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	129 trans = mail_index_transaction_begin(view,
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	130 MAIL_INDEX_TRANSACTION_FLAG_EXTERNAL);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	131 if (mail_cache_compress(cache, trans) < 0) {
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	132 mail_index_transaction_rollback(&trans);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	133 ret = -1;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	134 } else {
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	135 ret = mail_index_transaction_commit(&trans, &log_file_seq,
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	136 &log_file_offset);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	137 }
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	138 mail_index_view_close(&view);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	139 mail_cache_transaction_reset(ctx);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	140 return ret;
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	141 }
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	142
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	143 static void
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	144 mail_cache_transaction_open_if_needed(struct mail_cache_transaction_ctx *ctx)
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	145 {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	146 struct mail_cache *cache = ctx->cache;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	147 const struct mail_index_ext *ext;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	148 uint32_t idx;
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	149 int i;
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	150
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	151 if (!cache->opened) {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	152 (void)mail_cache_open_and_verify(cache);
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	153 return;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	154 }
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	155
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	156 /* see if we should try to reopen the cache file */
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	157 for (i = 0;; i++) {
7054 4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	158 if (MAIL_CACHE_IS_UNUSABLE(cache))
4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	159 return;
4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	160
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	161 if (!mail_index_map_get_ext_idx(cache->index->map,
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	162 cache->ext_id, &idx))
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	163 return;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	164
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	165 ext = array_idx(&cache->index->map->extensions, idx);
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	166 if (ext->reset_id == cache->hdr->file_seq \|\| i == 2)
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	167 break;
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	168
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	169 /* index offsets don't match the cache file */
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	170 if (ext->reset_id > cache->hdr->file_seq) {
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	171 /* the cache file appears to be too old.
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	172 reopening should help. */
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	173 if (mail_cache_reopen(cache) != 0)
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	174 break;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	175 }
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	176
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	177 /* cache file sequence might be broken. it's also possible
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	178 that it was just compressed and we just haven't yet seen
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	179 the changes in index. try if refreshing index helps.
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	180 if not, compress the cache file. */
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	181 if (i == 0) {
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	182 if (ctx->tried_compression)
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	183 break;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	184 /* get the latest reset ID */
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	185 if (mail_index_refresh(ctx->cache->index) < 0)
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	186 return;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	187 } else {
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	188 i_assert(i == 1);
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	189 (void)mail_cache_transaction_compress(ctx);
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	190 }
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	191 }
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	192 }
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	193
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	194 static int mail_cache_transaction_lock(struct mail_cache_transaction_ctx *ctx)
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	195 {
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	196 struct mail_cache *cache = ctx->cache;
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	197 int ret;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	198
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	199 mail_cache_transaction_open_if_needed(ctx);
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	200
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	201 if ((ret = mail_cache_lock(cache, FALSE)) <= 0) {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	202 if (ret < 0)
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	203 return -1;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	204
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	205 if (!ctx->tried_compression && MAIL_CACHE_IS_UNUSABLE(cache)) {
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	206 if (mail_cache_transaction_compress(ctx) < 0)
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	207 return -1;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	208 return mail_cache_transaction_lock(ctx);
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	209 } else {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	210 return 0;
6646 f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	211 }
4680 cd2db88396e1 Fix to delayed cache file opening, cached data might have been lost. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	212 }
cd2db88396e1 Fix to delayed cache file opening, cached data might have been lost. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	213
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	214 if (!MAIL_CACHE_IS_UNUSABLE(cache) && ctx->cache_file_seq == 0) {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	215 i_assert(ctx->cache_data == NULL \|\|
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	216 ctx->cache_data->used == 0);
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	217 ctx->cache_file_seq = cache->hdr->file_seq;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	218 } else if (ctx->cache_file_seq != cache->hdr->file_seq) {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	219 if (mail_cache_unlock(cache) < 0)
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	220 return -1;
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	221 mail_cache_transaction_reset(ctx);
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	222 return 0;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	223 }
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	224 return 1;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	225 }
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	226
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	227 static int mail_cache_grow_file(struct mail_cache *cache, size_t size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	228 {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	229 struct stat st;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	230 uoff_t new_fsize, grow_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	231
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	232 i_assert(cache->locked);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	233
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	234 /* grow the file */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	235 new_fsize = cache->hdr_copy.used_file_size + size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	236 grow_size = new_fsize / 100 * MAIL_CACHE_GROW_PERCENTAGE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	237 if (grow_size < 16384)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	238 grow_size = 16384;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	239 new_fsize += grow_size;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	240 new_fsize &= ~1023;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	241
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	242 if (fstat(cache->fd, &st) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	243 mail_cache_set_syscall_error(cache, "fstat()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	244 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	245 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	246
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	247 if ((uoff_t)st.st_size < new_fsize) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	248 if (file_set_size(cache->fd, new_fsize) < 0) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	249 mail_cache_set_syscall_error(cache, "file_set_size()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	250 return -1;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	251 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	252 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	253 return 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	254 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	255
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	256 static bool mail_cache_unlink_hole(struct mail_cache *cache, size_t size,
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	257 struct mail_cache_hole_header *hole_r)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	258 {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	259 struct mail_cache_header *hdr = &cache->hdr_copy;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	260 struct mail_cache_hole_header hole;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	261 uint32_t offset, prev_offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	262
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	263 i_assert(cache->locked);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	264
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	265 offset = hdr->hole_offset; prev_offset = 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	266 while (offset != 0) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	267 if (pread_full(cache->fd, &hole, sizeof(hole), offset) <= 0) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	268 mail_cache_set_syscall_error(cache, "pread_full()");
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	269 return FALSE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	270 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	271
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	272 if (hole.magic != MAIL_CACHE_HOLE_HEADER_MAGIC) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	273 mail_cache_set_corrupted(cache,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	274 "Invalid magic in hole header");
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	275 return FALSE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	276 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	278 if (hole.size >= size)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	279 break;
2747 1e65e0684377 Holes in cache file were broken Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	280
1e65e0684377 Holes in cache file were broken Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	281 prev_offset = offset;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	282 offset = hole.next_offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	283 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	284 if (offset == 0)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	285 return FALSE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	286
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	287 if (prev_offset == 0)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	288 hdr->hole_offset = hole.next_offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	289 else {
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	290 if (mail_cache_write(cache, &hole.next_offset,
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	291 sizeof(hole.next_offset), prev_offset) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	292 return FALSE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	293 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	294 hdr->deleted_space -= hole.size;
2747 1e65e0684377 Holes in cache file were broken Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	295 cache->hdr_modified = TRUE;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	296
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	297 hole_r->next_offset = offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	298 hole_r->size = hole.size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	299 return TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	300 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	301
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	302 static void
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	303 mail_cache_transaction_add_reservation(struct mail_cache_transaction_ctx *ctx,
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	304 uint32_t offset, uint32_t size)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	305 {
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	306 struct mail_cache_reservation res;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	307
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	308 ctx->reserved_space_offset = offset;
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	309 ctx->reserved_space = size;
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	310
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	311 res.offset = offset;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	312 res.size = size;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	313
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	314 array_append(&ctx->reservations, &res, 1);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	315 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	316
5389 00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	317 static void
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	318 mail_cache_transaction_partial_commit(struct mail_cache_transaction_ctx *ctx,
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	319 uint32_t offset, uint32_t size)
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	320 {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	321 struct mail_cache_reservation *res;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	322 unsigned int i, count;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	323
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	324 if (offset + size == ctx->cache->hdr_copy.used_file_size &&
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	325 offset + size == ctx->reserved_space_offset) {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	326 i_assert(ctx->reserved_space == 0);
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	327 ctx->reserved_space_offset = 0;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	328 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	329
5401 94ace1b42ad9 compile fix Timo Sirainen <tss@iki.fi> parents: 5389 diff changeset	330 res = array_get_modifiable(&ctx->reservations, &count);
5389 00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	331 for (i = 0; i < count; i++) {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	332 if (res[i].offset == offset) {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	333 if (res[i].size == size) {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	334 array_delete(&ctx->reservations, i, 1);
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	335 } else {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	336 i_assert(res[i].size > size);
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	337 res[i].offset += size;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	338 res[i].size -= size;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	339 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	340 break;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	341 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	342 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	343 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	344
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	345 static int
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	346 mail_cache_transaction_reserve_more(struct mail_cache_transaction_ctx *ctx,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	347 size_t block_size, bool commit)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	348 {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	349 struct mail_cache *cache = ctx->cache;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	350 struct mail_cache_header *hdr = &cache->hdr_copy;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	351 struct mail_cache_hole_header hole;
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	352 struct mail_cache_reservation *reservations;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	353 unsigned int count;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	354
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	355 i_assert(cache->locked);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	356
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	357 if (mail_cache_unlink_hole(cache, block_size, &hole)) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	358 /* found a large enough hole. */
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	359 mail_cache_transaction_add_reservation(ctx, hole.next_offset,
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	360 hole.size);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	361 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	362 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	363
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	364 if (MAIL_CACHE_IS_UNUSABLE(cache)) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	365 /* mail_cache_unlink_hole() could have noticed corruption */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	366 return -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	367 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	368
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	369 if ((uint32_t)-1 - hdr->used_file_size < block_size) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	370 mail_index_set_error(cache->index, "Cache file too large: %s",
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	371 cache->filepath);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	372 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	373 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	374
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	375 if (!commit && block_size < MAIL_CACHE_MAX_RESERVED_BLOCK_SIZE) {
2341 b3c0ed666757 Limit how much a single transaction can reserve space Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	376 /* allocate some more space than we need */
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	377 size_t new_block_size = (block_size + ctx->last_grow_size) * 2;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	378 if (new_block_size > MAIL_CACHE_MAX_RESERVED_BLOCK_SIZE)
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	379 new_block_size = MAIL_CACHE_MAX_RESERVED_BLOCK_SIZE;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	380
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	381 if ((uint32_t)-1 - hdr->used_file_size >= new_block_size) {
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	382 block_size = new_block_size;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	383 ctx->last_grow_size = new_block_size;
2341 b3c0ed666757 Limit how much a single transaction can reserve space Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	384 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	385 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	386
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	387 if (mail_cache_grow_file(ctx->cache, block_size) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	388 return -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	389
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	390 if (ctx->reserved_space_offset + ctx->reserved_space ==
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	391 hdr->used_file_size) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	392 /* we can simply grow it */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	393
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	394 /* grow reservation. it's probably the last one in the buffer,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	395 but it's not guarateed because we might have used holes
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	396 as well */
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4261 diff changeset	397 reservations = array_get_modifiable(&ctx->reservations, &count);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	398
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	399 do {
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	400 i_assert(count > 0);
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	401 count--;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	402 } while (reservations[count].offset +
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	403 reservations[count].size != hdr->used_file_size);
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	404
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	405 reservations[count].size += block_size;
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	406 ctx->reserved_space += block_size;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	407 } else {
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	408 mail_cache_transaction_add_reservation(ctx, hdr->used_file_size,
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	409 block_size);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	410 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	411
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	412 cache->hdr_modified = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	413 hdr->used_file_size = ctx->reserved_space_offset + ctx->reserved_space;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	414 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	415 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	416
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	417 static void
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	418 mail_cache_free_space(struct mail_cache *cache, uint32_t offset, uint32_t size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	419 {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	420 struct mail_cache_hole_header hole;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	421
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	422 i_assert(cache->locked);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	423
2914 37b81e395f00 Make sure we don't try to write to cache file when it's already closed Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	424 if (MAIL_CACHE_IS_UNUSABLE(cache))
37b81e395f00 Make sure we don't try to write to cache file when it's already closed Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	425 return;
37b81e395f00 Make sure we don't try to write to cache file when it's already closed Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	426
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	427 if (offset + size == cache->hdr_copy.used_file_size) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	428 /* we can just set used_file_size back */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	429 cache->hdr_modified = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	430 cache->hdr_copy.used_file_size = offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	431 } else if (size >= MAIL_CACHE_MIN_HOLE_SIZE) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	432 /* set it up as a hole */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	433 hole.next_offset = cache->hdr_copy.hole_offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	434 hole.size = size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	435 hole.magic = MAIL_CACHE_HOLE_HEADER_MAGIC;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	436
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	437 if (mail_cache_write(cache, &hole, sizeof(hole), offset) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	438 return;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	439
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	440 cache->hdr_copy.deleted_space += size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	441 cache->hdr_copy.hole_offset = offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	442 cache->hdr_modified = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	443 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	444 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	445
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	446 static int
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	447 mail_cache_transaction_free_space(struct mail_cache_transaction_ctx *ctx)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	448 {
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	449 bool locked = ctx->cache->locked;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	450
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	451 if (ctx->reserved_space == 0)
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	452 return 0;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	453
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	454 if (!locked) {
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	455 if (mail_cache_transaction_lock(ctx) <= 0)
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	456 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	457 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	458
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	459 /* check again - locking might have reopened the cache file */
db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	460 if (ctx->reserved_space != 0) {
2600 0f7327288390 extra asserts Timo Sirainen <tss@iki.fi> parents: 2598 diff changeset	461 i_assert(ctx->cache_file_seq == ctx->cache->hdr->file_seq);
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	462 mail_cache_free_space(ctx->cache, ctx->reserved_space_offset,
db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	463 ctx->reserved_space);
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	464 ctx->reserved_space_offset = 0;
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	465 ctx->reserved_space = 0;
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	466 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	467
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	468 if (!locked) {
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	469 if (mail_cache_unlock(ctx->cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	470 return -1;
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	471 }
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	472 return 0;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	473 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	474
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	475 static int
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	476 mail_cache_transaction_get_space(struct mail_cache_transaction_ctx *ctx,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	477 size_t min_size, size_t max_size,
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	478 uint32_t offset_r, size_t available_space_r,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	479 bool commit)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	480 {
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	481 bool locked = ctx->cache->locked;
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	482 uint32_t cache_file_seq;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	483 size_t size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	484 int ret;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	485
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	486 i_assert((min_size & 3) == 0);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	487 i_assert((max_size & 3) == 0);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	488
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	489 if (min_size > ctx->reserved_space) {
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	490 /* not enough preallocated space in transaction, get more */
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	491 cache_file_seq = ctx->cache_file_seq;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	492 if (!locked) {
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	493 if ((ret = mail_cache_transaction_lock(ctx)) <= 0)
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	494 return ret;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	495 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	496 ret = mail_cache_transaction_reserve_more(ctx, max_size,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	497 commit);
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	498 if (!locked) {
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	499 if (mail_cache_unlock(ctx->cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	500 return -1;
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	501 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	502
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	503 if (ret < 0)
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	504 return -1;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	505
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	506 if (cache_file_seq != ctx->cache_file_seq) {
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	507 /* cache file reopened - need to abort */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	508 return 0;
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	509 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	510
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	511 size = max_size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	512 } else {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	513 size = I_MIN(max_size, ctx->reserved_space);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	514 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	515
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	516 *offset_r = ctx->reserved_space_offset;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	517 ctx->reserved_space_offset += size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	518 ctx->reserved_space -= size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	519 if (available_space_r != NULL)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	520 *available_space_r = size;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	521 i_assert((size & 3) == 0);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	522
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	523 if (size == max_size && commit) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	524 /* final commit - see if we can free the rest of the
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	525 reserved space */
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	526 if (mail_cache_transaction_free_space(ctx) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	527 return -1;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	528 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	529
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	530 i_assert(size >= min_size);
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	531 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	532 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	533
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	534 static int
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	535 mail_cache_transaction_update_index(struct mail_cache_transaction_ctx *ctx,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	536 const struct mail_cache_record *rec,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	537 const uint32_t seq, uint32_t seq_idx,
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	538 uint32_t seq_limit, uint32_t write_offset,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	539 uint32_t *size_r)
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	540 {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	541 struct mail_cache *cache = ctx->cache;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	542 uint32_t i, old_offset, orig_write_offset;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	543
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	544 /* write the cache_offsets to index file. records' prev_offset
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	545 is updated to point to old cache record when index is being
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	546 synced. */
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	547 orig_write_offset = write_offset;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	548 for (i = *seq_idx; i < seq_limit; i++) {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	549 mail_index_update_ext(ctx->trans, seq[i], cache->ext_id,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	550 &write_offset, &old_offset);
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	551 if (old_offset != 0) {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	552 /* we added records for this message multiple
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	553 times in this same uncommitted transaction.
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	554 only the new one will be written to
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	555 transaction log, we need to do the linking
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	556 ourself here. */
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	557 if (old_offset > write_offset) {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	558 if (mail_cache_link_unlocked(cache, old_offset,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	559 write_offset) < 0)
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	560 return -1;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	561 } else {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	562 /* if we're combining multiple transactions,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	563 make sure the one with the smallest offset
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	564 is written into index. this is required for
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	565 non-file-mmaped cache to work properly. */
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	566 mail_index_update_ext(ctx->trans, seq[i],
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	567 cache->ext_id,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	568 &old_offset, NULL);
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	569 if (mail_cache_link_unlocked(cache,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	570 write_offset,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	571 old_offset) < 0)
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	572 return -1;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	573 }
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	574 }
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	575
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	576 write_offset += rec->size;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	577 rec = CONST_PTR_OFFSET(rec, rec->size);
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	578 }
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	579
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	580 *seq_idx = i;
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	581 *size_r = write_offset - orig_write_offset;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	582 return 0;
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	583 }
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	584
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	585 static int
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	586 mail_cache_transaction_flush(struct mail_cache_transaction_ctx *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	587 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	588 struct mail_cache *cache = ctx->cache;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	589 const struct mail_cache_record rec, tmp_rec;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	590 const uint32_t *seq;
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	591 uint32_t write_offset, write_size, rec_pos, seq_idx, seq_limit;
f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	592 size_t size, max_size;
f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	593 unsigned int seq_count;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	594 int ret;
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	595 bool commit;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	596
2578 9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	597 if (MAIL_CACHE_IS_UNUSABLE(cache))
9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	598 return -1;
9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	599
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	600 commit = ctx->prev_seq == 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	601 if (commit) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	602 /* committing, remove the last dummy record */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	603 buffer_set_used_size(ctx->cache_data, ctx->prev_pos);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	604 }
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	605
2598 2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	606 if (ctx->cache_file_seq != ctx->cache->hdr->file_seq) {
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	607 /* cache file reopened - need to abort */
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	608 mail_cache_transaction_reset(ctx);
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	609 return 0;
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	610 }
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	611
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	612 rec = buffer_get_data(ctx->cache_data, &size);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	613 i_assert(ctx->prev_pos <= size);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	614
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	615 seq = array_get(&ctx->cache_data_seq, &seq_count);
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	616 seq_limit = 0;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	617
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	618 for (seq_idx = 0, rec_pos = 0; rec_pos < ctx->prev_pos;) {
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	619 max_size = ctx->prev_pos - rec_pos;
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	620
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	621 ret = mail_cache_transaction_get_space(ctx, rec->size,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	622 max_size, &write_offset,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	623 &max_size, commit);
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	624 if (ret <= 0) {
5094 cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	625 /* error / couldn't lock / cache file reopened */
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	626 return ret;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	627 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	628
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	629 if (rec_pos + max_size < ctx->prev_pos) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	630 /* see how much we can really write there */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	631 tmp_rec = rec;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	632 for (size = 0; size + tmp_rec->size <= max_size; ) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	633 seq_limit++;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	634 size += tmp_rec->size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	635 tmp_rec = CONST_PTR_OFFSET(tmp_rec,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	636 tmp_rec->size);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	637 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	638 max_size = size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	639 } else {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	640 seq_limit = seq_count;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	641 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	642
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	643 /* write it to file */
2600 0f7327288390 extra asserts Timo Sirainen <tss@iki.fi> parents: 2598 diff changeset	644 i_assert(ctx->cache_file_seq == cache->hdr->file_seq);
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	645 if (mail_cache_write(cache, rec, max_size, write_offset) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	646 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	647
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	648 if (mail_cache_transaction_update_index(ctx, rec, seq,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	649 &seq_idx, seq_limit,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	650 write_offset,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	651 &write_size) < 0)
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	652 return -1;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	653
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	654 rec_pos += write_size;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	655 rec = CONST_PTR_OFFSET(rec, write_size);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	656 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	657
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	658 /* drop the written data from buffer */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	659 buffer_copy(ctx->cache_data, 0,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	660 ctx->cache_data, ctx->prev_pos, (size_t)-1);
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	661 buffer_set_used_size(ctx->cache_data,
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	662 buffer_get_used_size(ctx->cache_data) -
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	663 ctx->prev_pos);
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	664 ctx->prev_pos = 0;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	665
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	666 array_clear(&ctx->cache_data_seq);
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	667 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	668 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	669
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	670 static void
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	671 mail_cache_transaction_switch_seq(struct mail_cache_transaction_ctx *ctx)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	672 {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	673 struct mail_cache_record *rec, new_rec;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	674 void *data;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	675 size_t size;
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	676 uint32_t reset_id;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	677
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	678 if (ctx->prev_seq != 0) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	679 /* fix record size */
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4261 diff changeset	680 data = buffer_get_modifiable_data(ctx->cache_data, &size);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	681 rec = PTR_OFFSET(data, ctx->prev_pos);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	682 rec->size = size - ctx->prev_pos;
5094 cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	683 i_assert(rec->size > sizeof(*rec));
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	684
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	685 /* set prev_offset if possible */
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	686 rec->prev_offset =
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	687 mail_cache_lookup_cur_offset(ctx->view->view,
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	688 ctx->prev_seq, &reset_id);
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	689 if (reset_id != ctx->cache->hdr->file_seq)
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	690 rec->prev_offset = 0;
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	691 else
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	692 ctx->cache->hdr_copy.continued_record_count++;
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	693
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	694 array_append(&ctx->cache_data_seq, &ctx->prev_seq, 1);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	695 ctx->prev_pos = size;
2324 ea745d0c9c4b Delay memory allocations for cache transaction. We don't always need it. Timo Sirainen <tss@iki.fi> parents: 2315 diff changeset	696 } else if (ctx->cache_data == NULL) {
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	697 ctx->cache_data =
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	698 buffer_create_dynamic(default_pool,
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	699 MAIL_CACHE_WRITE_BUFFER);
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	700 i_array_init(&ctx->cache_data_seq, 64);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	701 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	702
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	703 memset(&new_rec, 0, sizeof(new_rec));
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	704 buffer_append(ctx->cache_data, &new_rec, sizeof(new_rec));
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	705
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	706 ctx->prev_seq = 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	707 ctx->changes = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	708 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	709
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	710 int mail_cache_transaction_commit(struct mail_cache_transaction_ctx **_ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	711 {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	712 struct mail_cache_transaction_ctx ctx = _ctx;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	713 struct mail_cache *cache = ctx->cache;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	714 int ret = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	715
2578 9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	716 if (!ctx->changes \|\| MAIL_CACHE_IS_UNUSABLE(cache)) {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	717 mail_cache_transaction_free(_ctx);
2201 7bdef5ea4591 Several fixes and cleanups to cache file code, still badly broken Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	718 return 0;
7bdef5ea4591 Several fixes and cleanups to cache file code, still badly broken Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	719 }
7bdef5ea4591 Several fixes and cleanups to cache file code, still badly broken Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	720
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	721 if (mail_cache_transaction_lock(ctx) <= 0) {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	722 mail_cache_transaction_rollback(_ctx);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	723 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	724 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	725
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	726 if (ctx->prev_seq != 0)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	727 mail_cache_transaction_switch_seq(ctx);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	728
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	729 if (mail_cache_transaction_flush(ctx) < 0)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	730 ret = -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	731
3808 8ec5b5955235 Don't fdatasync() cache file. It's not all that important. Timo Sirainen <tss@iki.fi> parents: 3688 diff changeset	732 /* Here would be a good place to do fdatasync() to make sure
8ec5b5955235 Don't fdatasync() cache file. It's not all that important. Timo Sirainen <tss@iki.fi> parents: 3688 diff changeset	733 everything is written before offsets are updated to index.
8ec5b5955235 Don't fdatasync() cache file. It's not all that important. Timo Sirainen <tss@iki.fi> parents: 3688 diff changeset	734 However it slows down I/O unneededly and we're pretty good at
8ec5b5955235 Don't fdatasync() cache file. It's not all that important. Timo Sirainen <tss@iki.fi> parents: 3688 diff changeset	735 catching and fixing cache corruption, so we no longer do it. */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	736
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	737 if (mail_cache_unlock(cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	738 ret = -1;
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	739 mail_cache_transaction_free(_ctx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	740 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	741 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	742
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	743 void mail_cache_transaction_rollback(struct mail_cache_transaction_ctx **_ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	744 {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	745 struct mail_cache_transaction_ctx ctx = _ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	746 struct mail_cache *cache = ctx->cache;
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	747 const struct mail_cache_reservation *reservations;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	748 unsigned int count;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	749
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	750 if ((ctx->reserved_space > 0 \|\| array_count(&ctx->reservations) > 0) &&
2578 9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	751 !MAIL_CACHE_IS_UNUSABLE(cache)) {
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	752 if (mail_cache_transaction_lock(ctx) > 0) {
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	753 reservations = array_get(&ctx->reservations, &count);
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	754
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	755 /* free flushed data as well. do it from end to
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	756 beginning so we have a better chance of
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	757 updating used_file_size instead of adding
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	758 holes */
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	759 while (count > 0) {
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	760 count--;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	761 mail_cache_free_space(ctx->cache,
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	762 reservations[count].offset,
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	763 reservations[count].size);
2420 90f25ed2dcef rollback: don't lock file if we don't need to Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	764 }
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	765 (void)mail_cache_unlock(cache);
2315 22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	766 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	767 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	768
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	769 mail_cache_transaction_free(_ctx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	770 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	771
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	772 static int
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	773 mail_cache_header_fields_write(struct mail_cache_transaction_ctx *ctx,
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	774 const buffer_t *buffer)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	775 {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	776 struct mail_cache *cache = ctx->cache;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	777 size_t size = buffer->used;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	778 uint32_t offset, hdr_offset;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	779
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	780 if (mail_cache_transaction_get_space(ctx, size, size,
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	781 &offset, NULL, TRUE) <= 0)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	782 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	783
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	784 if (mail_cache_write(cache, buffer->data, size, offset) < 0)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	785 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	786
6735 a881755b3db8 We don't really care if we lost a cache file or if it gets broken, so don't Timo Sirainen <tss@iki.fi> parents: 6733 diff changeset	787 if (cache->index->nfs_flush) {
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	788 if (fdatasync(cache->fd) < 0) {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	789 mail_cache_set_syscall_error(cache, "fdatasync()");
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	790 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	791 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	792 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	793 if (mail_cache_header_fields_get_next_offset(cache, &hdr_offset) < 0)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	794 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	795
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	796 /* if we rollback the transaction, we must not overwrite this
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	797 area because it's already committed after updating the
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	798 header offset */
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	799 mail_cache_transaction_partial_commit(ctx, offset, size);
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	800
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	801 /* after it's guaranteed to be in disk, update header offset */
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	802 offset = mail_index_uint32_to_offset(offset);
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	803 if (mail_cache_write(cache, &offset, sizeof(offset), hdr_offset) < 0)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	804 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	805
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	806 if (hdr_offset == offsetof(struct mail_cache_header,
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	807 field_header_offset)) {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	808 /* we're adding the first field. hdr_copy needs to be kept
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	809 in sync so unlocking won't overwrite it. */
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	810 cache->hdr_copy.field_header_offset = hdr_offset;
6703 75c48f171ad3 Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise Timo Sirainen <tss@iki.fi> parents: 6699 diff changeset	811 cache->hdr_ro_copy.field_header_offset = hdr_offset;
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	812 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	813 return 0;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	814 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	815
7105 f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	816 static void mail_cache_mark_adding(struct mail_cache *cache, bool set)
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	817 {
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	818 unsigned int i;
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	819
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	820 /* we want to avoid adding all the fields one by one to the cache file,
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	821 so just add all of them at once in here. the unused ones get dropped
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	822 later when compressing. */
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	823 for (i = 0; i < cache->fields_count; i++) {
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	824 if (set)
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	825 cache->fields[i].used = TRUE;
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	826 cache->fields[i].adding = set;
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	827 }
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	828 }
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	829
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	830 static int mail_cache_header_add_field(struct mail_cache_transaction_ctx *ctx,
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	831 unsigned int field_idx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	832 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	833 struct mail_cache *cache = ctx->cache;
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	834 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	835
5947 7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	836 if ((ret = mail_cache_transaction_lock(ctx)) <= 0) {
6731 42acb5629519 Crashfix in error conditions Timo Sirainen <tss@iki.fi> parents: 6709 diff changeset	837 if (MAIL_CACHE_IS_UNUSABLE(cache))
42acb5629519 Crashfix in error conditions Timo Sirainen <tss@iki.fi> parents: 6709 diff changeset	838 return -1;
42acb5629519 Crashfix in error conditions Timo Sirainen <tss@iki.fi> parents: 6709 diff changeset	839
6699 faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	840 /* if we compressed the cache, the field should be there now.
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	841 it's however possible that someone else just compressed it
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	842 and we only reopened the cache file. */
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	843 if (cache->field_file_map[field_idx] != (uint32_t)-1)
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	844 return 0;
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	845
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	846 /* need to add it */
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	847 if ((ret = mail_cache_transaction_lock(ctx)) <= 0)
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	848 return -1;
5947 7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	849 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	850
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	851 /* re-read header to make sure we don't lose any fields. */
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	852 if (mail_cache_header_fields_read(cache) < 0) {
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	853 (void)mail_cache_unlock(cache);
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	854 return -1;
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	855 }
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	856
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	857 if (cache->field_file_map[field_idx] != (uint32_t)-1) {
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	858 /* it was already added */
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	859 if (mail_cache_unlock(cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	860 return -1;
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	861 return 0;
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	862 }
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	863
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	864 T_FRAME(
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	865 buffer_t *buffer;
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	866
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	867 buffer = buffer_create_dynamic(pool_datastack_create(), 256);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	868 mail_cache_header_fields_get(cache, buffer);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	869 ret = mail_cache_header_fields_write(ctx, buffer);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	870 );
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	871
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	872 if (ret == 0) {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	873 /* we wrote all the headers, so there are no pending changes */
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	874 cache->field_header_write_pending = FALSE;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	875 ret = mail_cache_header_fields_read(cache);
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	876 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	877 if (ret == 0 && cache->field_file_map[field_idx] == (uint32_t)-1) {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	878 mail_index_set_error(cache->index,
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	879 "Cache file %s: Newly added field got "
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	880 "lost unexpectedly", cache->filepath);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	881 ret = -1;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	882 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	883
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	884 if (mail_cache_unlock(cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	885 ret = -1;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	886 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	887 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	888
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	889 void mail_cache_add(struct mail_cache_transaction_ctx *ctx, uint32_t seq,
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	890 unsigned int field_idx, const void *data, size_t data_size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	891 {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	892 uint32_t file_field, data_size32;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	893 unsigned int fixed_size;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	894 size_t full_size;
7105 f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	895 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	896
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	897 i_assert(field_idx < ctx->cache->fields_count);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	898 i_assert(data_size < (uint32_t)-1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	899
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	900 if (ctx->cache->fields[field_idx].field.decision ==
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	901 (MAIL_CACHE_DECISION_NO \| MAIL_CACHE_DECISION_FORCED))
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	902 return;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	903
6646 f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	904 if (ctx->cache_file_seq == 0) {
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	905 mail_cache_transaction_open_if_needed(ctx);
6655 9ffa528c257c Last change broke cache file creation completely. Timo Sirainen <tss@iki.fi> parents: 6646 diff changeset	906 if (!MAIL_CACHE_IS_UNUSABLE(ctx->cache))
9ffa528c257c Last change broke cache file creation completely. Timo Sirainen <tss@iki.fi> parents: 6646 diff changeset	907 ctx->cache_file_seq = ctx->cache->hdr->file_seq;
6733 72dad50b1c9f Crashfix if cache file got unusable in the middle of a transaction. Timo Sirainen <tss@iki.fi> parents: 6731 diff changeset	908 } else if (!MAIL_CACHE_IS_UNUSABLE(ctx->cache) &&
72dad50b1c9f Crashfix if cache file got unusable in the middle of a transaction. Timo Sirainen <tss@iki.fi> parents: 6731 diff changeset	909 ctx->cache_file_seq != ctx->cache->hdr->file_seq) {
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	910 /* cache was compressed within this transaction */
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	911 mail_cache_transaction_reset(ctx);
6646 f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	912 }
f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	913
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	914 file_field = ctx->cache->field_file_map[field_idx];
6667 b6c76ea86e2a Assert-crashfix Timo Sirainen <tss@iki.fi> parents: 6655 diff changeset	915 if (MAIL_CACHE_IS_UNUSABLE(ctx->cache) \|\| file_field == (uint32_t)-1) {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	916 /* we'll have to add this field to headers */
7105 f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	917 mail_cache_mark_adding(ctx->cache, TRUE);
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	918 ret = mail_cache_header_add_field(ctx, field_idx);
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	919 mail_cache_mark_adding(ctx->cache, FALSE);
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	920 if (ret < 0)
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	921 return;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	922
6696 68adf46a6b3f When adding new fields to the cache file, add all the fields we currently Timo Sirainen <tss@iki.fi> parents: 6667 diff changeset	923 if (ctx->cache_file_seq == 0)
68adf46a6b3f When adding new fields to the cache file, add all the fields we currently Timo Sirainen <tss@iki.fi> parents: 6667 diff changeset	924 ctx->cache_file_seq = ctx->cache->hdr->file_seq;
68adf46a6b3f When adding new fields to the cache file, add all the fields we currently Timo Sirainen <tss@iki.fi> parents: 6667 diff changeset	925
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	926 file_field = ctx->cache->field_file_map[field_idx];
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	927 i_assert(file_field != (uint32_t)-1);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	928 }
6667 b6c76ea86e2a Assert-crashfix Timo Sirainen <tss@iki.fi> parents: 6655 diff changeset	929 i_assert(ctx->cache_file_seq != 0);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	930
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	931 mail_cache_decision_add(ctx->view, seq, field_idx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	932
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	933 fixed_size = ctx->cache->fields[field_idx].field.field_size;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	934 i_assert(fixed_size == (unsigned int)-1 \|\| fixed_size == data_size);
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	935
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	936 data_size32 = (uint32_t)data_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	937
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	938 if (ctx->prev_seq != seq) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	939 mail_cache_transaction_switch_seq(ctx);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	940 ctx->prev_seq = seq;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	941
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	942 /* remember roughly what we have modified, so cache lookups can
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	943 look into transactions to see changes. */
2315 22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	944 if (seq < ctx->view->trans_seq1 \|\| ctx->view->trans_seq1 == 0)
22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	945 ctx->view->trans_seq1 = seq;
22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	946 if (seq > ctx->view->trans_seq2)
22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	947 ctx->view->trans_seq2 = seq;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	948 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	949
6004 6645e902b8a0 mail_cache_add(): Update existing fields array so mail_cache_field_exists() Timo Sirainen <tss@iki.fi> parents: 5966 diff changeset	950 /* remember that this value exists, in case we try to look it up */
6645e902b8a0 mail_cache_add(): Update existing fields array so mail_cache_field_exists() Timo Sirainen <tss@iki.fi> parents: 5966 diff changeset	951 buffer_write(ctx->view->cached_exists_buf, field_idx,
6645e902b8a0 mail_cache_add(): Update existing fields array so mail_cache_field_exists() Timo Sirainen <tss@iki.fi> parents: 5966 diff changeset	952 &ctx->view->cached_exists_value, 1);
6645e902b8a0 mail_cache_add(): Update existing fields array so mail_cache_field_exists() Timo Sirainen <tss@iki.fi> parents: 5966 diff changeset	953
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	954 full_size = (data_size + 3) & ~3;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	955 if (fixed_size == (unsigned int)-1)
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	956 full_size += sizeof(data_size32);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	957
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	958 if (ctx->cache_data->used + full_size >
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	959 buffer_get_size(ctx->cache_data) && ctx->prev_pos > 0) {
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	960 /* time to flush our buffer. if flushing fails because the
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	961 cache file had been compressed and was reopened, return
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	962 without adding the cached data since cache_data buffer
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	963 doesn't contain the cache_rec anymore. */
5094 cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	964 if (mail_cache_transaction_flush(ctx) <= 0) {
cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	965 /* make sure the transaction is reset, so we don't
cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	966 constantly try to flush for each call to this
cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	967 function */
cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	968 mail_cache_transaction_reset(ctx);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	969 return;
5094 cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	970 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	971 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	972
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	973 buffer_append(ctx->cache_data, &file_field, sizeof(file_field));
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	974 if (fixed_size == (unsigned int)-1) {
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	975 buffer_append(ctx->cache_data, &data_size32,
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	976 sizeof(data_size32));
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	977 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	978
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	979 buffer_append(ctx->cache_data, data, data_size);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	980 if ((data_size & 3) != 0)
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	981 buffer_append_zero(ctx->cache_data, 4 - (data_size & 3));
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	982 }
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	983
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	984 bool mail_cache_field_want_add(struct mail_cache_transaction_ctx *ctx,
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	985 uint32_t seq, unsigned int field_idx)
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	986 {
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	987 enum mail_cache_decision_type decision;
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	988
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	989 mail_cache_transaction_open_if_needed(ctx);
5966 731be87a2128 Make sure cache is opened when mail_cache_field_can_add() or Timo Sirainen <tss@iki.fi> parents: 5960 diff changeset	990
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	991 decision = mail_cache_field_get_decision(ctx->view->cache, field_idx);
6841 0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	992 decision &= ~MAIL_CACHE_DECISION_FORCED;
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	993 switch (decision) {
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	994 case MAIL_CACHE_DECISION_NO:
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	995 return FALSE;
6841 0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	996 case MAIL_CACHE_DECISION_TEMP:
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	997 /* add it only if it's newer than what we would drop when
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	998 compressing */
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	999 if (ctx->first_new_seq == 0) {
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1000 ctx->first_new_seq =
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1001 mail_cache_get_first_new_seq(ctx->view->view);
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1002 }
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1003 if (seq < ctx->first_new_seq)
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1004 return FALSE;
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1005 break;
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1006 default:
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1007 break;
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1008 }
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1009
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	1010 return mail_cache_field_exists(ctx->view, seq, field_idx) == 0;
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1011 }
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1012
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1013 bool mail_cache_field_can_add(struct mail_cache_transaction_ctx *ctx,
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	1014 uint32_t seq, unsigned int field_idx)
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1015 {
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1016 enum mail_cache_decision_type decision;
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1017
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	1018 mail_cache_transaction_open_if_needed(ctx);
5966 731be87a2128 Make sure cache is opened when mail_cache_field_can_add() or Timo Sirainen <tss@iki.fi> parents: 5960 diff changeset	1019
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	1020 decision = mail_cache_field_get_decision(ctx->view->cache, field_idx);
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1021 if (decision == (MAIL_CACHE_DECISION_FORCED \| MAIL_CACHE_DECISION_NO))
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1022 return FALSE;
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1023
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	1024 return mail_cache_field_exists(ctx->view, seq, field_idx) == 0;
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1025 }
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1026
2354 1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1027 static int mail_cache_link_unlocked(struct mail_cache *cache,
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1028 uint32_t old_offset, uint32_t new_offset)
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1029 {
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1030 new_offset += offsetof(struct mail_cache_record, prev_offset);
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	1031 return mail_cache_write(cache, &old_offset, sizeof(old_offset),
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	1032 new_offset);
2354 1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1033 }
2315 22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	1034
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1035 int mail_cache_link(struct mail_cache *cache, uint32_t old_offset,
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1036 uint32_t new_offset)
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1037 {
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1038 const struct mail_cache_record *rec;
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1039
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1040 i_assert(cache->locked);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1041
2578 9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	1042 if (MAIL_CACHE_IS_UNUSABLE(cache))
9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	1043 return -1;
9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	1044
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1045 /* this function is called for each added cache record (or cache
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1046 extension record update actually) with new_offset pointing to the
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1047 new record and old_offset pointing to the previous record.
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1048
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1049 we want to keep the old and new records linked so both old and new
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1050 cached data is found. normally they are already linked correctly.
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1051 the problem only comes when multiple processes are adding cache
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1052 records at the same time. we'd rather not lose those additions, so
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1053 force the linking order to be new_offset -> old_offset if it isn't
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1054 already. */
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1055 if (mail_cache_map(cache, new_offset, sizeof(*rec)) < 0)
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1056 return -1;
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1057 if (new_offset + sizeof(*rec) > cache->mmap_length) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1058 mail_cache_set_corrupted(cache,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1059 "Cache record offset %u points outside file",
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1060 new_offset);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1061 return -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1062 }
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1063 rec = CACHE_RECORD(cache, new_offset);
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1064 if (rec->prev_offset == old_offset) {
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1065 /* link is already correct */
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1066 return 0;
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1067 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1068
2354 1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1069 if (mail_cache_link_unlocked(cache, old_offset, new_offset) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1070 return -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1071
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1072 cache->hdr_copy.continued_record_count++;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1073 cache->hdr_modified = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1074 return 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1075 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1076
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1077 static int mail_cache_delete_real(struct mail_cache *cache, uint32_t offset)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1078 {
5734 5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1079 const struct mail_cache_record *rec;
5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1080 ARRAY_TYPE(uint32_t) looping_offsets;
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1081
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1082 i_assert(cache->locked);
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1083
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1084 /* we'll only update the deleted_space in header. we can't really
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1085 do any actual deleting as other processes might still be using
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1086 the data. also it's actually useful as some index views are still
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1087 able to ask cached data from messages that have already been
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1088 expunged. */
5734 5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1089 t_array_init(&looping_offsets, 8);
5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1090 array_append(&looping_offsets, &offset, 1);
5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1091 while (mail_cache_get_record(cache, offset, &rec) == 0) {
5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1092 cache->hdr_copy.deleted_space += rec->size;
5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1093 offset = rec->prev_offset;
5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1094
5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1095 if (offset == 0) {
5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1096 /* successfully got to the end of the list */
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1097 return 0;
3688 1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1098 }
1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1099
5734 5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1100 if (mail_cache_track_loops(&looping_offsets, offset)) {
3688 1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1101 mail_cache_set_corrupted(cache,
1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1102 "record list is circular");
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1103 return -1;
3688 1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1104 }
1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1105 }
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1106 return -1;
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1107 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1108
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1109 int mail_cache_delete(struct mail_cache *cache, uint32_t offset)
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1110 {
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1111 int ret;
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1112
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1113 T_FRAME(
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1114 ret = mail_cache_delete_real(cache, offset);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1115 );
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1116 cache->hdr_modified = TRUE;
5734 5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1117 return ret;
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1118 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-cache-transaction.c @ 7138:876c7bca351c HEAD