dovecot/original-hg/dovecot-1.2: src/lib-index/mail-cache-transaction.c annotate

annotate src/lib-index/mail-cache-transaction.c @ 9458:adee8cb3ff5d HEAD

Minor memory allocation tweaks.

author	Timo Sirainen <tss@iki.fi>
date	Wed, 28 Oct 2009 14:10:04 -0400
parents	b9faf4db2a9f
children	00cd9aacd03c

rev	line source
8590 b9faf4db2a9f Updated copyright notices to include year 2009. Timo Sirainen <tss@iki.fi> parents: 8273 diff changeset	1 /* Copyright (c) 2003-2009 Dovecot authors, see the included COPYING file */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
6296 205ee38f10d1 Drop fields that haven't been used for 30 days when compressing. Timo Sirainen <tss@iki.fi> parents: 6004 diff changeset	4 #include "ioloop.h"
3470 346a494c2feb Moved array declaration to array-decl.h and include it in lib.h. So array.h Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	5 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	6 #include "buffer.h"
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	7 #include "file-cache.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	8 #include "file-set-size.h"
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	9 #include "read-full.h"
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	10 #include "write-full.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	11 #include "mail-cache-private.h"
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	12 #include "mail-index-transaction-private.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	13
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2247 diff changeset	14 #include <stddef.h>
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	15 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	16
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	17 #define MAIL_CACHE_WRITE_BUFFER 32768
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	18
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	19 struct mail_cache_reservation {
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	20 uint32_t offset;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	21 uint32_t size;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	22 };
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	23
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	24 struct mail_cache_transaction_ctx {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	25 struct mail_cache *cache;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	26 struct mail_cache_view *view;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	27 struct mail_index_transaction *trans;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	28
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	29 uint32_t cache_file_seq;
6841 0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	30 uint32_t first_new_seq;
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	31
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	32 buffer_t *cache_data;
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4261 diff changeset	33 ARRAY_DEFINE(cache_data_seq, uint32_t);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	34 uint32_t prev_seq;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	35 size_t prev_pos;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	36
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4261 diff changeset	37 ARRAY_DEFINE(reservations, struct mail_cache_reservation);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	38 uint32_t reserved_space_offset, reserved_space;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	39 uint32_t last_grow_size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	40
5947 7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	41 unsigned int tried_compression:1;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	42 unsigned int changes:1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	43 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	44
2354 1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	45 static int mail_cache_link_unlocked(struct mail_cache *cache,
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	46 uint32_t old_offset, uint32_t new_offset);
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	47
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	48 struct mail_cache_transaction_ctx *
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	49 mail_cache_get_transaction(struct mail_cache_view *view,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	50 struct mail_index_transaction *t)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	51 {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	52 struct mail_cache_transaction_ctx *ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	53
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	54 if (t->cache_trans_ctx != NULL)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	55 return t->cache_trans_ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	56
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	57 ctx = i_new(struct mail_cache_transaction_ctx, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 ctx->cache = view->cache;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	59 ctx->view = view;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	60 ctx->trans = t;
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	61 i_array_init(&ctx->reservations, 32);
2597 18cddf71996a Don't crash when cache isn't usable. Timo Sirainen <tss@iki.fi> parents: 2596 diff changeset	62
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	63 i_assert(view->transaction == NULL);
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	64 view->transaction = ctx;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2747 diff changeset	65 view->trans_view = mail_index_transaction_open_updated_view(t);
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	66
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	67 t->cache_trans_ctx = ctx;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	68 return ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	69 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	70
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	71 static void mail_cache_transaction_reset(struct mail_cache_transaction_ctx *ctx)
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	72 {
5181 922672fcdfdc Crashfix in error conditions. Timo Sirainen <tss@iki.fi> parents: 5099 diff changeset	73 ctx->cache_file_seq = MAIL_CACHE_IS_UNUSABLE(ctx->cache) ? 0 :
922672fcdfdc Crashfix in error conditions. Timo Sirainen <tss@iki.fi> parents: 5099 diff changeset	74 ctx->cache->hdr->file_seq;
5947 7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	75 mail_index_ext_set_reset_id(ctx->trans, ctx->cache->ext_id,
7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	76 ctx->cache_file_seq);
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	77
6646 f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	78 if (ctx->cache_data != NULL)
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	79 buffer_set_used_size(ctx->cache_data, 0);
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	80 if (array_is_created(&ctx->cache_data_seq))
f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	81 array_clear(&ctx->cache_data_seq);
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	82 ctx->prev_seq = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	83 ctx->prev_pos = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	84
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	85 array_clear(&ctx->reservations);
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	86 ctx->reserved_space_offset = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	87 ctx->reserved_space = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	88 ctx->last_grow_size = 0;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	89
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	90 ctx->changes = FALSE;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	91 }
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	92
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	93 static void
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	94 mail_cache_transaction_free(struct mail_cache_transaction_ctx **_ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	95 {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	96 struct mail_cache_transaction_ctx ctx = _ctx;
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	97
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	98 *_ctx = NULL;
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	99
5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	100 ctx->trans->cache_trans_ctx = NULL;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	101 ctx->view->transaction = NULL;
2315 22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	102 ctx->view->trans_seq1 = ctx->view->trans_seq2 = 0;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	103
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	104 mail_index_view_close(&ctx->view->trans_view);
2324 ea745d0c9c4b Delay memory allocations for cache transaction. We don't always need it. Timo Sirainen <tss@iki.fi> parents: 2315 diff changeset	105 if (ctx->cache_data != NULL)
6414 a6a49d5efc59 Changed buffer_free() and buffer_free_without_data() APIs to take ** pointer Timo Sirainen <tss@iki.fi> parents: 6296 diff changeset	106 buffer_free(&ctx->cache_data);
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	107 if (array_is_created(&ctx->cache_data_seq))
f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	108 array_free(&ctx->cache_data_seq);
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	109 array_free(&ctx->reservations);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	110 i_free(ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	111 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	112
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	113 static int
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	114 mail_cache_transaction_compress(struct mail_cache_transaction_ctx *ctx)
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	115 {
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	116 struct mail_cache *cache = ctx->cache;
6831 ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	117 struct mail_index_view *view;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	118 struct mail_index_transaction *trans;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	119 uint32_t log_file_seq;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	120 uoff_t log_file_offset;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	121 int ret;
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	122
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	123 ctx->tried_compression = TRUE;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	124
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	125 cache->need_compress_file_seq =
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	126 MAIL_CACHE_IS_UNUSABLE(cache) ? 0 : cache->hdr->file_seq;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	127
6831 ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	128 view = mail_index_view_open(cache->index);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	129 trans = mail_index_transaction_begin(view,
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	130 MAIL_INDEX_TRANSACTION_FLAG_EXTERNAL);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	131 if (mail_cache_compress(cache, trans) < 0) {
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	132 mail_index_transaction_rollback(&trans);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	133 ret = -1;
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	134 } else {
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	135 ret = mail_index_transaction_commit(&trans, &log_file_seq,
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	136 &log_file_offset);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	137 }
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	138 mail_index_view_close(&view);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	139 mail_cache_transaction_reset(ctx);
ddf393049f28 When we create the initial cache file or recreate a broken one, write the Timo Sirainen <tss@iki.fi> parents: 6830 diff changeset	140 return ret;
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	141 }
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	142
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	143 static void
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	144 mail_cache_transaction_open_if_needed(struct mail_cache_transaction_ctx *ctx)
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	145 {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	146 struct mail_cache *cache = ctx->cache;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	147 const struct mail_index_ext *ext;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	148 uint32_t idx;
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	149 int i;
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	150
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	151 if (!cache->opened) {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	152 (void)mail_cache_open_and_verify(cache);
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	153 return;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	154 }
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	155
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	156 /* see if we should try to reopen the cache file */
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	157 for (i = 0;; i++) {
7054 4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	158 if (MAIL_CACHE_IS_UNUSABLE(cache))
4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	159 return;
4c6e9edfd4ce Fixed crashes at error conditions. Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	160
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	161 if (!mail_index_map_get_ext_idx(cache->index->map,
8273 40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	162 cache->ext_id, &idx)) {
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	163 /* index doesn't have a cache extension, but the cache
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	164 file exists (corrupted indexes fixed?). fix it. */
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	165 if (i == 2)
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	166 break;
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	167 } else {
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	168 ext = array_idx(&cache->index->map->extensions, idx);
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	169 if (ext->reset_id == cache->hdr->file_seq \|\| i == 2)
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	170 break;
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	171
8273 40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	172 /* index offsets don't match the cache file */
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	173 if (ext->reset_id > cache->hdr->file_seq) {
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	174 /* the cache file appears to be too old.
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	175 reopening should help. */
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	176 if (mail_cache_reopen(cache) != 0)
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	177 break;
40df4d4cced0 Cache file was never updated if the file existed but index didn't have cache extension. Timo Sirainen <tss@iki.fi> parents: 7312 diff changeset	178 }
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	179 }
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	180
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	181 /* cache file sequence might be broken. it's also possible
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	182 that it was just compressed and we just haven't yet seen
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	183 the changes in index. try if refreshing index helps.
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	184 if not, compress the cache file. */
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	185 if (i == 0) {
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	186 if (ctx->tried_compression)
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	187 break;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	188 /* get the latest reset ID */
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	189 if (mail_index_refresh(ctx->cache->index) < 0)
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	190 return;
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	191 } else {
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	192 i_assert(i == 1);
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	193 (void)mail_cache_transaction_compress(ctx);
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	194 }
3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	195 }
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	196 }
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	197
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	198 static int mail_cache_transaction_lock(struct mail_cache_transaction_ctx *ctx)
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	199 {
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	200 struct mail_cache *cache = ctx->cache;
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	201 int ret;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	202
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	203 mail_cache_transaction_open_if_needed(ctx);
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	204
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	205 if ((ret = mail_cache_lock(cache, FALSE)) <= 0) {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	206 if (ret < 0)
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	207 return -1;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	208
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	209 if (!ctx->tried_compression && MAIL_CACHE_IS_UNUSABLE(cache)) {
6830 3a41003bb1ae If cache file's file_seq appears to be broken, handle it by compressing the Timo Sirainen <tss@iki.fi> parents: 6735 diff changeset	210 if (mail_cache_transaction_compress(ctx) < 0)
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	211 return -1;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	212 return mail_cache_transaction_lock(ctx);
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	213 } else {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	214 return 0;
6646 f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	215 }
4680 cd2db88396e1 Fix to delayed cache file opening, cached data might have been lost. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	216 }
cd2db88396e1 Fix to delayed cache file opening, cached data might have been lost. Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	217
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	218 if (!MAIL_CACHE_IS_UNUSABLE(cache) && ctx->cache_file_seq == 0) {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	219 i_assert(ctx->cache_data == NULL \|\|
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	220 ctx->cache_data->used == 0);
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	221 ctx->cache_file_seq = cache->hdr->file_seq;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	222 } else if (ctx->cache_file_seq != cache->hdr->file_seq) {
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	223 if (mail_cache_unlock(cache) < 0)
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	224 return -1;
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	225 mail_cache_transaction_reset(ctx);
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	226 return 0;
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	227 }
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	228 return 1;
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	229 }
6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	230
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	231 static int mail_cache_grow_file(struct mail_cache *cache, size_t size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	232 {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	233 struct stat st;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	234 uoff_t new_fsize, grow_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	235
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	236 i_assert(cache->locked);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	237
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	238 /* grow the file */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	239 new_fsize = cache->hdr_copy.used_file_size + size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	240 grow_size = new_fsize / 100 * MAIL_CACHE_GROW_PERCENTAGE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	241 if (grow_size < 16384)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	242 grow_size = 16384;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	243 new_fsize += grow_size;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	244 new_fsize &= ~1023;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	245
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	246 if (fstat(cache->fd, &st) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	247 mail_cache_set_syscall_error(cache, "fstat()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	248 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	249 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	250
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	251 if ((uoff_t)st.st_size < new_fsize) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	252 if (file_set_size(cache->fd, new_fsize) < 0) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	253 mail_cache_set_syscall_error(cache, "file_set_size()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	254 return -1;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	255 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	256 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	257 return 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	258 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	259
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	260 static bool mail_cache_unlink_hole(struct mail_cache *cache, size_t size,
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	261 struct mail_cache_hole_header *hole_r)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	262 {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	263 struct mail_cache_header *hdr = &cache->hdr_copy;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	264 struct mail_cache_hole_header hole;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	265 uint32_t offset, prev_offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	266
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	267 i_assert(cache->locked);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	268
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	269 offset = hdr->hole_offset; prev_offset = 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	270 while (offset != 0) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	271 if (pread_full(cache->fd, &hole, sizeof(hole), offset) <= 0) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	272 mail_cache_set_syscall_error(cache, "pread_full()");
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	273 return FALSE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	274 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	275
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	276 if (hole.magic != MAIL_CACHE_HOLE_HEADER_MAGIC) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	277 mail_cache_set_corrupted(cache,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	278 "Invalid magic in hole header");
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	279 return FALSE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	280 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	281
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	282 if (hole.size >= size)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	283 break;
2747 1e65e0684377 Holes in cache file were broken Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	284
1e65e0684377 Holes in cache file were broken Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	285 prev_offset = offset;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	286 offset = hole.next_offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	287 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	288 if (offset == 0)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	289 return FALSE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	290
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	291 if (prev_offset == 0)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	292 hdr->hole_offset = hole.next_offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	293 else {
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	294 if (mail_cache_write(cache, &hole.next_offset,
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	295 sizeof(hole.next_offset), prev_offset) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	296 return FALSE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	297 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	298 hdr->deleted_space -= hole.size;
2747 1e65e0684377 Holes in cache file were broken Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	299 cache->hdr_modified = TRUE;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	300
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	301 hole_r->next_offset = offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	302 hole_r->size = hole.size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	303 return TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	304 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	305
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	306 static void
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	307 mail_cache_transaction_add_reservation(struct mail_cache_transaction_ctx *ctx,
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	308 uint32_t offset, uint32_t size)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	309 {
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	310 struct mail_cache_reservation res;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	311
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	312 ctx->reserved_space_offset = offset;
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	313 ctx->reserved_space = size;
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	314
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	315 res.offset = offset;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	316 res.size = size;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	317
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	318 array_append(&ctx->reservations, &res, 1);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	319 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	320
5389 00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	321 static void
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	322 mail_cache_transaction_partial_commit(struct mail_cache_transaction_ctx *ctx,
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	323 uint32_t offset, uint32_t size)
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	324 {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	325 struct mail_cache_reservation *res;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	326 unsigned int i, count;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	327
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	328 if (offset + size == ctx->cache->hdr_copy.used_file_size &&
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	329 offset + size == ctx->reserved_space_offset) {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	330 i_assert(ctx->reserved_space == 0);
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	331 ctx->reserved_space_offset = 0;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	332 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	333
5401 94ace1b42ad9 compile fix Timo Sirainen <tss@iki.fi> parents: 5389 diff changeset	334 res = array_get_modifiable(&ctx->reservations, &count);
5389 00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	335 for (i = 0; i < count; i++) {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	336 if (res[i].offset == offset) {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	337 if (res[i].size == size) {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	338 array_delete(&ctx->reservations, i, 1);
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	339 } else {
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	340 i_assert(res[i].size > size);
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	341 res[i].offset += size;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	342 res[i].size -= size;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	343 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	344 break;
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	345 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	346 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	347 }
00645e05ae96 If we update the fields list in header, it's committed immediately so if the Timo Sirainen <tss@iki.fi> parents: 5181 diff changeset	348
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	349 static int
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	350 mail_cache_transaction_reserve_more(struct mail_cache_transaction_ctx *ctx,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	351 size_t block_size, bool commit)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	352 {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	353 struct mail_cache *cache = ctx->cache;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	354 struct mail_cache_header *hdr = &cache->hdr_copy;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	355 struct mail_cache_hole_header hole;
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	356 struct mail_cache_reservation *reservations;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	357 unsigned int count;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	358
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	359 i_assert(cache->locked);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	360
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	361 if (mail_cache_unlink_hole(cache, block_size, &hole)) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	362 /* found a large enough hole. */
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	363 mail_cache_transaction_add_reservation(ctx, hole.next_offset,
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	364 hole.size);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	365 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	366 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	367
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	368 if (MAIL_CACHE_IS_UNUSABLE(cache)) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	369 /* mail_cache_unlink_hole() could have noticed corruption */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	370 return -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	371 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	372
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	373 if ((uint32_t)-1 - hdr->used_file_size < block_size) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	374 mail_index_set_error(cache->index, "Cache file too large: %s",
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	375 cache->filepath);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	376 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	377 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	378
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	379 if (!commit && block_size < MAIL_CACHE_MAX_RESERVED_BLOCK_SIZE) {
2341 b3c0ed666757 Limit how much a single transaction can reserve space Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	380 /* allocate some more space than we need */
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	381 size_t new_block_size = (block_size + ctx->last_grow_size) * 2;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	382 if (new_block_size > MAIL_CACHE_MAX_RESERVED_BLOCK_SIZE)
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	383 new_block_size = MAIL_CACHE_MAX_RESERVED_BLOCK_SIZE;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	384
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	385 if ((uint32_t)-1 - hdr->used_file_size >= new_block_size) {
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	386 block_size = new_block_size;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	387 ctx->last_grow_size = new_block_size;
2341 b3c0ed666757 Limit how much a single transaction can reserve space Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	388 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	389 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	390
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	391 if (mail_cache_grow_file(ctx->cache, block_size) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	392 return -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	393
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	394 if (ctx->reserved_space_offset + ctx->reserved_space ==
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	395 hdr->used_file_size) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	396 /* we can simply grow it */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	397
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	398 /* grow reservation. it's probably the last one in the buffer,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	399 but it's not guarateed because we might have used holes
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	400 as well */
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4261 diff changeset	401 reservations = array_get_modifiable(&ctx->reservations, &count);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	402
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	403 do {
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	404 i_assert(count > 0);
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	405 count--;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	406 } while (reservations[count].offset +
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	407 reservations[count].size != hdr->used_file_size);
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	408
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	409 reservations[count].size += block_size;
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	410 ctx->reserved_space += block_size;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	411 } else {
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	412 mail_cache_transaction_add_reservation(ctx, hdr->used_file_size,
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	413 block_size);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	414 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	415
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	416 cache->hdr_modified = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	417 hdr->used_file_size = ctx->reserved_space_offset + ctx->reserved_space;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	418 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	419 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	420
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	421 static void
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	422 mail_cache_free_space(struct mail_cache *cache, uint32_t offset, uint32_t size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	423 {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	424 struct mail_cache_hole_header hole;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	425
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	426 i_assert(cache->locked);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	427
2914 37b81e395f00 Make sure we don't try to write to cache file when it's already closed Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	428 if (MAIL_CACHE_IS_UNUSABLE(cache))
37b81e395f00 Make sure we don't try to write to cache file when it's already closed Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	429 return;
37b81e395f00 Make sure we don't try to write to cache file when it's already closed Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	430
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	431 if (offset + size == cache->hdr_copy.used_file_size) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	432 /* we can just set used_file_size back */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	433 cache->hdr_modified = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	434 cache->hdr_copy.used_file_size = offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	435 } else if (size >= MAIL_CACHE_MIN_HOLE_SIZE) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	436 /* set it up as a hole */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	437 hole.next_offset = cache->hdr_copy.hole_offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	438 hole.size = size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	439 hole.magic = MAIL_CACHE_HOLE_HEADER_MAGIC;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	440
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	441 if (mail_cache_write(cache, &hole, sizeof(hole), offset) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	442 return;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	443
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	444 cache->hdr_copy.deleted_space += size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	445 cache->hdr_copy.hole_offset = offset;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	446 cache->hdr_modified = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	447 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	448 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	449
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	450 static int
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	451 mail_cache_transaction_free_space(struct mail_cache_transaction_ctx *ctx)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	452 {
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	453 bool locked = ctx->cache->locked;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	454
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	455 if (ctx->reserved_space == 0)
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	456 return 0;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	457
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	458 if (!locked) {
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	459 if (mail_cache_transaction_lock(ctx) <= 0)
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	460 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	461 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	462
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	463 /* check again - locking might have reopened the cache file */
db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	464 if (ctx->reserved_space != 0) {
2600 0f7327288390 extra asserts Timo Sirainen <tss@iki.fi> parents: 2598 diff changeset	465 i_assert(ctx->cache_file_seq == ctx->cache->hdr->file_seq);
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	466 mail_cache_free_space(ctx->cache, ctx->reserved_space_offset,
db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	467 ctx->reserved_space);
3186 31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	468 ctx->reserved_space_offset = 0;
31313e1fe036 Space reservation fixes and cleanups. Timo Sirainen <tss@iki.fi> parents: 3175 diff changeset	469 ctx->reserved_space = 0;
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	470 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	471
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	472 if (!locked) {
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	473 if (mail_cache_unlock(ctx->cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	474 return -1;
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	475 }
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	476 return 0;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	477 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	478
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	479 static int
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	480 mail_cache_transaction_get_space(struct mail_cache_transaction_ctx *ctx,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	481 size_t min_size, size_t max_size,
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	482 uint32_t offset_r, size_t available_space_r,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	483 bool commit)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	484 {
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	485 bool locked = ctx->cache->locked;
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	486 uint32_t cache_file_seq;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	487 size_t size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	488 int ret;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	489
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	490 i_assert((min_size & 3) == 0);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	491 i_assert((max_size & 3) == 0);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	492
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	493 if (min_size > ctx->reserved_space) {
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	494 /* not enough preallocated space in transaction, get more */
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	495 cache_file_seq = ctx->cache_file_seq;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	496 if (!locked) {
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	497 if ((ret = mail_cache_transaction_lock(ctx)) <= 0)
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	498 return ret;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	499 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	500 ret = mail_cache_transaction_reserve_more(ctx, max_size,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	501 commit);
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	502 if (!locked) {
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	503 if (mail_cache_unlock(ctx->cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	504 return -1;
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	505 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	506
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	507 if (ret < 0)
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	508 return -1;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	509
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	510 if (cache_file_seq != ctx->cache_file_seq) {
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	511 /* cache file reopened - need to abort */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	512 return 0;
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	513 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	514
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	515 size = max_size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	516 } else {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	517 size = I_MIN(max_size, ctx->reserved_space);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	518 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	519
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	520 *offset_r = ctx->reserved_space_offset;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	521 ctx->reserved_space_offset += size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	522 ctx->reserved_space -= size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	523 if (available_space_r != NULL)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	524 *available_space_r = size;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	525 i_assert((size & 3) == 0);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	526
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	527 if (size == max_size && commit) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	528 /* final commit - see if we can free the rest of the
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	529 reserved space */
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	530 if (mail_cache_transaction_free_space(ctx) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	531 return -1;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	532 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	533
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	534 i_assert(size >= min_size);
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	535 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	536 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	537
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	538 static int
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	539 mail_cache_transaction_update_index(struct mail_cache_transaction_ctx *ctx,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	540 const struct mail_cache_record *rec,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	541 const uint32_t seq, uint32_t seq_idx,
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	542 uint32_t seq_limit, uint32_t write_offset,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	543 uint32_t *size_r)
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	544 {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	545 struct mail_cache *cache = ctx->cache;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	546 uint32_t i, old_offset, orig_write_offset;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	547
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	548 /* write the cache_offsets to index file. records' prev_offset
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	549 is updated to point to old cache record when index is being
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	550 synced. */
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	551 orig_write_offset = write_offset;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	552 for (i = *seq_idx; i < seq_limit; i++) {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	553 mail_index_update_ext(ctx->trans, seq[i], cache->ext_id,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	554 &write_offset, &old_offset);
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	555 if (old_offset != 0) {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	556 /* we added records for this message multiple
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	557 times in this same uncommitted transaction.
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	558 only the new one will be written to
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	559 transaction log, we need to do the linking
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	560 ourself here. */
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	561 if (old_offset > write_offset) {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	562 if (mail_cache_link_unlocked(cache, old_offset,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	563 write_offset) < 0)
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	564 return -1;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	565 } else {
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	566 /* if we're combining multiple transactions,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	567 make sure the one with the smallest offset
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	568 is written into index. this is required for
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	569 non-file-mmaped cache to work properly. */
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	570 mail_index_update_ext(ctx->trans, seq[i],
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	571 cache->ext_id,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	572 &old_offset, NULL);
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	573 if (mail_cache_link_unlocked(cache,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	574 write_offset,
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	575 old_offset) < 0)
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	576 return -1;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	577 }
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	578 }
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	579
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	580 write_offset += rec->size;
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	581 rec = CONST_PTR_OFFSET(rec, rec->size);
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	582 }
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	583
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	584 *seq_idx = i;
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	585 *size_r = write_offset - orig_write_offset;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	586 return 0;
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	587 }
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	588
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	589 static int
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	590 mail_cache_transaction_flush(struct mail_cache_transaction_ctx *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	591 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	592 struct mail_cache *cache = ctx->cache;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	593 const struct mail_cache_record rec, tmp_rec;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	594 const uint32_t *seq;
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	595 uint32_t write_offset, write_size, rec_pos, seq_idx, seq_limit;
f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	596 size_t size, max_size;
f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	597 unsigned int seq_count;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	598 int ret;
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3808 diff changeset	599 bool commit;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	600
2578 9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	601 if (MAIL_CACHE_IS_UNUSABLE(cache))
9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	602 return -1;
9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	603
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	604 commit = ctx->prev_seq == 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	605 if (commit) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	606 /* committing, remove the last dummy record */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	607 buffer_set_used_size(ctx->cache_data, ctx->prev_pos);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	608 }
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	609
2598 2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	610 if (ctx->cache_file_seq != ctx->cache->hdr->file_seq) {
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	611 /* cache file reopened - need to abort */
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	612 mail_cache_transaction_reset(ctx);
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	613 return 0;
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	614 }
2568be07706f one more "cache file reopened" fix. Timo Sirainen <tss@iki.fi> parents: 2597 diff changeset	615
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	616 rec = buffer_get_data(ctx->cache_data, &size);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	617 i_assert(ctx->prev_pos <= size);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	618
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	619 seq = array_get(&ctx->cache_data_seq, &seq_count);
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	620 seq_limit = 0;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	621
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	622 for (seq_idx = 0, rec_pos = 0; rec_pos < ctx->prev_pos;) {
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	623 max_size = ctx->prev_pos - rec_pos;
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	624
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	625 ret = mail_cache_transaction_get_space(ctx, rec->size,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	626 max_size, &write_offset,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	627 &max_size, commit);
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	628 if (ret <= 0) {
5094 cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	629 /* error / couldn't lock / cache file reopened */
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	630 return ret;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	631 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	632
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	633 if (rec_pos + max_size < ctx->prev_pos) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	634 /* see how much we can really write there */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	635 tmp_rec = rec;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	636 for (size = 0; size + tmp_rec->size <= max_size; ) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	637 seq_limit++;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	638 size += tmp_rec->size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	639 tmp_rec = CONST_PTR_OFFSET(tmp_rec,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	640 tmp_rec->size);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	641 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	642 max_size = size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	643 } else {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	644 seq_limit = seq_count;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	645 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	646
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	647 /* write it to file */
2600 0f7327288390 extra asserts Timo Sirainen <tss@iki.fi> parents: 2598 diff changeset	648 i_assert(ctx->cache_file_seq == cache->hdr->file_seq);
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	649 if (mail_cache_write(cache, rec, max_size, write_offset) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	650 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	651
3137 0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	652 if (mail_cache_transaction_update_index(ctx, rec, seq,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	653 &seq_idx, seq_limit,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	654 write_offset,
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	655 &write_size) < 0)
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	656 return -1;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	657
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	658 rec_pos += write_size;
0383923d4759 cleanups / minor fixes Timo Sirainen <tss@iki.fi> parents: 3000 diff changeset	659 rec = CONST_PTR_OFFSET(rec, write_size);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	660 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	661
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	662 /* drop the written data from buffer */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	663 buffer_copy(ctx->cache_data, 0,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	664 ctx->cache_data, ctx->prev_pos, (size_t)-1);
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	665 buffer_set_used_size(ctx->cache_data,
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	666 buffer_get_used_size(ctx->cache_data) -
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	667 ctx->prev_pos);
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	668 ctx->prev_pos = 0;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	669
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	670 array_clear(&ctx->cache_data_seq);
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	671 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	672 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	673
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	674 static void
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	675 mail_cache_transaction_switch_seq(struct mail_cache_transaction_ctx *ctx)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	676 {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	677 struct mail_cache_record *rec, new_rec;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	678 void *data;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	679 size_t size;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	680
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	681 if (ctx->prev_seq != 0) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	682 /* fix record size */
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4261 diff changeset	683 data = buffer_get_modifiable_data(ctx->cache_data, &size);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	684 rec = PTR_OFFSET(data, ctx->prev_pos);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	685 rec->size = size - ctx->prev_pos;
5094 cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	686 i_assert(rec->size > sizeof(*rec));
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	687
7199 24266b9a6c55 Removed cache prev_offset optimization for now. It could have caused cache Timo Sirainen <tss@iki.fi> parents: 7138 diff changeset	688 /* FIXME: here would be a good place to set prev_offset to
24266b9a6c55 Removed cache prev_offset optimization for now. It could have caused cache Timo Sirainen <tss@iki.fi> parents: 7138 diff changeset	689 avoid doing it later, but avoid circular prev_offsets
24266b9a6c55 Removed cache prev_offset optimization for now. It could have caused cache Timo Sirainen <tss@iki.fi> parents: 7138 diff changeset	690 when cache is updated multiple times within the same
24266b9a6c55 Removed cache prev_offset optimization for now. It could have caused cache Timo Sirainen <tss@iki.fi> parents: 7138 diff changeset	691 transaction */
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	692
3277 f022867fb58d buffer -> array Timo Sirainen <tss@iki.fi> parents: 3186 diff changeset	693 array_append(&ctx->cache_data_seq, &ctx->prev_seq, 1);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	694 ctx->prev_pos = size;
2324 ea745d0c9c4b Delay memory allocations for cache transaction. We don't always need it. Timo Sirainen <tss@iki.fi> parents: 2315 diff changeset	695 } else if (ctx->cache_data == NULL) {
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	696 ctx->cache_data =
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	697 buffer_create_dynamic(default_pool,
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	698 MAIL_CACHE_WRITE_BUFFER);
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	699 i_array_init(&ctx->cache_data_seq, 64);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	700 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	701
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	702 memset(&new_rec, 0, sizeof(new_rec));
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	703 buffer_append(ctx->cache_data, &new_rec, sizeof(new_rec));
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	704
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	705 ctx->prev_seq = 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	706 ctx->changes = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	707 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	708
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	709 int mail_cache_transaction_commit(struct mail_cache_transaction_ctx **_ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	710 {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	711 struct mail_cache_transaction_ctx ctx = _ctx;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	712 struct mail_cache *cache = ctx->cache;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	713 int ret = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	714
2578 9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	715 if (!ctx->changes \|\| MAIL_CACHE_IS_UNUSABLE(cache)) {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	716 mail_cache_transaction_free(_ctx);
2201 7bdef5ea4591 Several fixes and cleanups to cache file code, still badly broken Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	717 return 0;
7bdef5ea4591 Several fixes and cleanups to cache file code, still badly broken Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	718 }
7bdef5ea4591 Several fixes and cleanups to cache file code, still badly broken Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	719
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	720 if (mail_cache_transaction_lock(ctx) <= 0) {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	721 mail_cache_transaction_rollback(_ctx);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	722 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	723 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	724
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	725 if (ctx->prev_seq != 0)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	726 mail_cache_transaction_switch_seq(ctx);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	727
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	728 if (mail_cache_transaction_flush(ctx) < 0)
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	729 ret = -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	730
3808 8ec5b5955235 Don't fdatasync() cache file. It's not all that important. Timo Sirainen <tss@iki.fi> parents: 3688 diff changeset	731 /* Here would be a good place to do fdatasync() to make sure
8ec5b5955235 Don't fdatasync() cache file. It's not all that important. Timo Sirainen <tss@iki.fi> parents: 3688 diff changeset	732 everything is written before offsets are updated to index.
8ec5b5955235 Don't fdatasync() cache file. It's not all that important. Timo Sirainen <tss@iki.fi> parents: 3688 diff changeset	733 However it slows down I/O unneededly and we're pretty good at
8ec5b5955235 Don't fdatasync() cache file. It's not all that important. Timo Sirainen <tss@iki.fi> parents: 3688 diff changeset	734 catching and fixing cache corruption, so we no longer do it. */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	735
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	736 if (mail_cache_unlock(cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	737 ret = -1;
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	738 mail_cache_transaction_free(_ctx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	739 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	740 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	741
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	742 void mail_cache_transaction_rollback(struct mail_cache_transaction_ctx **_ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	743 {
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	744 struct mail_cache_transaction_ctx ctx = _ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	745 struct mail_cache *cache = ctx->cache;
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	746 const struct mail_cache_reservation *reservations;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	747 unsigned int count;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	748
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	749 if ((ctx->reserved_space > 0 \|\| array_count(&ctx->reservations) > 0) &&
2578 9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	750 !MAIL_CACHE_IS_UNUSABLE(cache)) {
2593 6c03f22b2489 If cache file was recreated the buffered transaction changes were written Timo Sirainen <tss@iki.fi> parents: 2586 diff changeset	751 if (mail_cache_transaction_lock(ctx) > 0) {
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	752 reservations = array_get(&ctx->reservations, &count);
2596 db059dc5d5c8 More fixes to make sure we don't write buffered data to cache file after Timo Sirainen <tss@iki.fi> parents: 2593 diff changeset	753
3953 d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	754 /* free flushed data as well. do it from end to
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	755 beginning so we have a better chance of
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	756 updating used_file_size instead of adding
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	757 holes */
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	758 while (count > 0) {
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	759 count--;
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	760 mail_cache_free_space(ctx->cache,
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	761 reservations[count].offset,
d9a7ad93ddde Changed reservations buffer to array. Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	762 reservations[count].size);
2420 90f25ed2dcef rollback: don't lock file if we don't need to Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	763 }
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	764 (void)mail_cache_unlock(cache);
2315 22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	765 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	766 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	767
6455 5ad7a36ca8c3 mail_cache_transaction_commit()/rollback() API is now public and takes a ** Timo Sirainen <tss@iki.fi> parents: 6443 diff changeset	768 mail_cache_transaction_free(_ctx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	769 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	770
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	771 static int
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	772 mail_cache_header_fields_write(struct mail_cache_transaction_ctx *ctx,
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	773 const buffer_t *buffer)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	774 {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	775 struct mail_cache *cache = ctx->cache;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	776 size_t size = buffer->used;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	777 uint32_t offset, hdr_offset;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	778
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	779 if (mail_cache_transaction_get_space(ctx, size, size,
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	780 &offset, NULL, TRUE) <= 0)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	781 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	782
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	783 if (mail_cache_write(cache, buffer->data, size, offset) < 0)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	784 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	785
6735 a881755b3db8 We don't really care if we lost a cache file or if it gets broken, so don't Timo Sirainen <tss@iki.fi> parents: 6733 diff changeset	786 if (cache->index->nfs_flush) {
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	787 if (fdatasync(cache->fd) < 0) {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	788 mail_cache_set_syscall_error(cache, "fdatasync()");
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	789 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	790 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	791 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	792 if (mail_cache_header_fields_get_next_offset(cache, &hdr_offset) < 0)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	793 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	794
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	795 /* if we rollback the transaction, we must not overwrite this
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	796 area because it's already committed after updating the
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	797 header offset */
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	798 mail_cache_transaction_partial_commit(ctx, offset, size);
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	799
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	800 /* after it's guaranteed to be in disk, update header offset */
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	801 offset = mail_index_uint32_to_offset(offset);
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	802 if (mail_cache_write(cache, &offset, sizeof(offset), hdr_offset) < 0)
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	803 return -1;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	804
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	805 if (hdr_offset == offsetof(struct mail_cache_header,
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	806 field_header_offset)) {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	807 /* we're adding the first field. hdr_copy needs to be kept
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	808 in sync so unlocking won't overwrite it. */
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	809 cache->hdr_copy.field_header_offset = hdr_offset;
6703 75c48f171ad3 Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise Timo Sirainen <tss@iki.fi> parents: 6699 diff changeset	810 cache->hdr_ro_copy.field_header_offset = hdr_offset;
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	811 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	812 return 0;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	813 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	814
7105 f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	815 static void mail_cache_mark_adding(struct mail_cache *cache, bool set)
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	816 {
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	817 unsigned int i;
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	818
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	819 /* we want to avoid adding all the fields one by one to the cache file,
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	820 so just add all of them at once in here. the unused ones get dropped
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	821 later when compressing. */
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	822 for (i = 0; i < cache->fields_count; i++) {
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	823 if (set)
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	824 cache->fields[i].used = TRUE;
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	825 cache->fields[i].adding = set;
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	826 }
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	827 }
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	828
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	829 static int mail_cache_header_add_field(struct mail_cache_transaction_ctx *ctx,
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	830 unsigned int field_idx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	831 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	832 struct mail_cache *cache = ctx->cache;
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	833 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	834
5947 7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	835 if ((ret = mail_cache_transaction_lock(ctx)) <= 0) {
6731 42acb5629519 Crashfix in error conditions Timo Sirainen <tss@iki.fi> parents: 6709 diff changeset	836 if (MAIL_CACHE_IS_UNUSABLE(cache))
42acb5629519 Crashfix in error conditions Timo Sirainen <tss@iki.fi> parents: 6709 diff changeset	837 return -1;
42acb5629519 Crashfix in error conditions Timo Sirainen <tss@iki.fi> parents: 6709 diff changeset	838
6699 faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	839 /* if we compressed the cache, the field should be there now.
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	840 it's however possible that someone else just compressed it
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	841 and we only reopened the cache file. */
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	842 if (cache->field_file_map[field_idx] != (uint32_t)-1)
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	843 return 0;
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	844
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	845 /* need to add it */
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	846 if ((ret = mail_cache_transaction_lock(ctx)) <= 0)
faa4da358d37 Last change caused assert-crashes. Timo Sirainen <tss@iki.fi> parents: 6696 diff changeset	847 return -1;
5947 7ebe0593f488 Don't create cache file until something is actually being added to it. Timo Sirainen <tss@iki.fi> parents: 5735 diff changeset	848 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	849
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	850 /* re-read header to make sure we don't lose any fields. */
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	851 if (mail_cache_header_fields_read(cache) < 0) {
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	852 (void)mail_cache_unlock(cache);
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	853 return -1;
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	854 }
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	855
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	856 if (cache->field_file_map[field_idx] != (uint32_t)-1) {
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	857 /* it was already added */
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	858 if (mail_cache_unlock(cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	859 return -1;
2586 6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	860 return 0;
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	861 }
6bff38d73dd3 Before adding new fields to cache file, make sure they weren't already Timo Sirainen <tss@iki.fi> parents: 2578 diff changeset	862
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7199 diff changeset	863 T_BEGIN {
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	864 buffer_t *buffer;
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	865
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	866 buffer = buffer_create_dynamic(pool_datastack_create(), 256);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	867 mail_cache_header_fields_get(cache, buffer);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	868 ret = mail_cache_header_fields_write(ctx, buffer);
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7199 diff changeset	869 } T_END;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	870
6443 53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	871 if (ret == 0) {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	872 /* we wrote all the headers, so there are no pending changes */
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	873 cache->field_header_write_pending = FALSE;
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	874 ret = mail_cache_header_fields_read(cache);
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	875 }
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	876 if (ret == 0 && cache->field_file_map[field_idx] == (uint32_t)-1) {
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	877 mail_index_set_error(cache->index,
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	878 "Cache file %s: Newly added field got "
53313d1e7e4f Code cleanups and some minor fixes. Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	879 "lost unexpectedly", cache->filepath);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	880 ret = -1;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	881 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	882
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	883 if (mail_cache_unlock(cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3560 diff changeset	884 ret = -1;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	885 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	886 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	887
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	888 void mail_cache_add(struct mail_cache_transaction_ctx *ctx, uint32_t seq,
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	889 unsigned int field_idx, const void *data, size_t data_size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	890 {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	891 uint32_t file_field, data_size32;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	892 unsigned int fixed_size;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	893 size_t full_size;
7105 f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	894 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	895
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	896 i_assert(field_idx < ctx->cache->fields_count);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	897 i_assert(data_size < (uint32_t)-1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	898
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	899 if (ctx->cache->fields[field_idx].field.decision ==
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	900 (MAIL_CACHE_DECISION_NO \| MAIL_CACHE_DECISION_FORCED))
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	901 return;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	902
6646 f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	903 if (ctx->cache_file_seq == 0) {
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	904 mail_cache_transaction_open_if_needed(ctx);
6655 9ffa528c257c Last change broke cache file creation completely. Timo Sirainen <tss@iki.fi> parents: 6646 diff changeset	905 if (!MAIL_CACHE_IS_UNUSABLE(ctx->cache))
9ffa528c257c Last change broke cache file creation completely. Timo Sirainen <tss@iki.fi> parents: 6646 diff changeset	906 ctx->cache_file_seq = ctx->cache->hdr->file_seq;
6733 72dad50b1c9f Crashfix if cache file got unusable in the middle of a transaction. Timo Sirainen <tss@iki.fi> parents: 6731 diff changeset	907 } else if (!MAIL_CACHE_IS_UNUSABLE(ctx->cache) &&
72dad50b1c9f Crashfix if cache file got unusable in the middle of a transaction. Timo Sirainen <tss@iki.fi> parents: 6731 diff changeset	908 ctx->cache_file_seq != ctx->cache->hdr->file_seq) {
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	909 /* cache was compressed within this transaction */
395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	910 mail_cache_transaction_reset(ctx);
6646 f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	911 }
f3ca0b9e3628 Fixed a race condition causing cache corruption when creating/compressing Timo Sirainen <tss@iki.fi> parents: 6589 diff changeset	912
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	913 file_field = ctx->cache->field_file_map[field_idx];
6667 b6c76ea86e2a Assert-crashfix Timo Sirainen <tss@iki.fi> parents: 6655 diff changeset	914 if (MAIL_CACHE_IS_UNUSABLE(ctx->cache) \|\| file_field == (uint32_t)-1) {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	915 /* we'll have to add this field to headers */
7105 f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	916 mail_cache_mark_adding(ctx->cache, TRUE);
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	917 ret = mail_cache_header_add_field(ctx, field_idx);
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	918 mail_cache_mark_adding(ctx->cache, FALSE);
f0ad529ac9ea Fixes to handling when fields are dropped from cache file. Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	919 if (ret < 0)
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	920 return;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	921
6696 68adf46a6b3f When adding new fields to the cache file, add all the fields we currently Timo Sirainen <tss@iki.fi> parents: 6667 diff changeset	922 if (ctx->cache_file_seq == 0)
68adf46a6b3f When adding new fields to the cache file, add all the fields we currently Timo Sirainen <tss@iki.fi> parents: 6667 diff changeset	923 ctx->cache_file_seq = ctx->cache->hdr->file_seq;
68adf46a6b3f When adding new fields to the cache file, add all the fields we currently Timo Sirainen <tss@iki.fi> parents: 6667 diff changeset	924
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	925 file_field = ctx->cache->field_file_map[field_idx];
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	926 i_assert(file_field != (uint32_t)-1);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	927 }
6667 b6c76ea86e2a Assert-crashfix Timo Sirainen <tss@iki.fi> parents: 6655 diff changeset	928 i_assert(ctx->cache_file_seq != 0);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	929
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	930 mail_cache_decision_add(ctx->view, seq, field_idx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	931
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	932 fixed_size = ctx->cache->fields[field_idx].field.field_size;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	933 i_assert(fixed_size == (unsigned int)-1 \|\| fixed_size == data_size);
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	934
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	935 data_size32 = (uint32_t)data_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	936
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	937 if (ctx->prev_seq != seq) {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	938 mail_cache_transaction_switch_seq(ctx);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	939 ctx->prev_seq = seq;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	940
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	941 /* remember roughly what we have modified, so cache lookups can
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	942 look into transactions to see changes. */
2315 22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	943 if (seq < ctx->view->trans_seq1 \|\| ctx->view->trans_seq1 == 0)
22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	944 ctx->view->trans_seq1 = seq;
22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	945 if (seq > ctx->view->trans_seq2)
22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	946 ctx->view->trans_seq2 = seq;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	947 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	948
6004 6645e902b8a0 mail_cache_add(): Update existing fields array so mail_cache_field_exists() Timo Sirainen <tss@iki.fi> parents: 5966 diff changeset	949 /* remember that this value exists, in case we try to look it up */
6645e902b8a0 mail_cache_add(): Update existing fields array so mail_cache_field_exists() Timo Sirainen <tss@iki.fi> parents: 5966 diff changeset	950 buffer_write(ctx->view->cached_exists_buf, field_idx,
6645e902b8a0 mail_cache_add(): Update existing fields array so mail_cache_field_exists() Timo Sirainen <tss@iki.fi> parents: 5966 diff changeset	951 &ctx->view->cached_exists_value, 1);
6645e902b8a0 mail_cache_add(): Update existing fields array so mail_cache_field_exists() Timo Sirainen <tss@iki.fi> parents: 5966 diff changeset	952
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	953 full_size = (data_size + 3) & ~3;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	954 if (fixed_size == (unsigned int)-1)
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	955 full_size += sizeof(data_size32);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	956
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	957 if (ctx->cache_data->used + full_size >
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	958 buffer_get_size(ctx->cache_data) && ctx->prev_pos > 0) {
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	959 /* time to flush our buffer. if flushing fails because the
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	960 cache file had been compressed and was reopened, return
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	961 without adding the cached data since cache_data buffer
3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	962 doesn't contain the cache_rec anymore. */
5094 cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	963 if (mail_cache_transaction_flush(ctx) <= 0) {
cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	964 /* make sure the transaction is reset, so we don't
cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	965 constantly try to flush for each call to this
cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	966 function */
cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	967 mail_cache_transaction_reset(ctx);
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	968 return;
5094 cce9bab229d7 If cache file wasn't opened yet when flushing a transaction, the changes Timo Sirainen <tss@iki.fi> parents: 4918 diff changeset	969 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	970 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	971
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2324 diff changeset	972 buffer_append(ctx->cache_data, &file_field, sizeof(file_field));
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	973 if (fixed_size == (unsigned int)-1) {
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	974 buffer_append(ctx->cache_data, &data_size32,
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	975 sizeof(data_size32));
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	976 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	977
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	978 buffer_append(ctx->cache_data, data, data_size);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	979 if ((data_size & 3) != 0)
3560 3533923e786f We could have written corrupted data to cache file if it had just been Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	980 buffer_append_zero(ctx->cache_data, 4 - (data_size & 3));
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	981 }
2200 97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	982
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	983 bool mail_cache_field_want_add(struct mail_cache_transaction_ctx *ctx,
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	984 uint32_t seq, unsigned int field_idx)
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	985 {
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	986 enum mail_cache_decision_type decision;
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	987
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	988 mail_cache_transaction_open_if_needed(ctx);
5966 731be87a2128 Make sure cache is opened when mail_cache_field_can_add() or Timo Sirainen <tss@iki.fi> parents: 5960 diff changeset	989
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	990 decision = mail_cache_field_get_decision(ctx->view->cache, field_idx);
6841 0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	991 decision &= ~MAIL_CACHE_DECISION_FORCED;
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	992 switch (decision) {
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	993 case MAIL_CACHE_DECISION_NO:
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	994 return FALSE;
6841 0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	995 case MAIL_CACHE_DECISION_TEMP:
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	996 /* add it only if it's newer than what we would drop when
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	997 compressing */
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	998 if (ctx->first_new_seq == 0) {
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	999 ctx->first_new_seq =
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1000 mail_cache_get_first_new_seq(ctx->view->view);
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1001 }
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1002 if (seq < ctx->first_new_seq)
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1003 return FALSE;
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1004 break;
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1005 default:
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1006 break;
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're Timo Sirainen <tss@iki.fi> parents: 6831 diff changeset	1007 }
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1008
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	1009 return mail_cache_field_exists(ctx->view, seq, field_idx) == 0;
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1010 }
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1011
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1012 bool mail_cache_field_can_add(struct mail_cache_transaction_ctx *ctx,
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	1013 uint32_t seq, unsigned int field_idx)
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1014 {
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1015 enum mail_cache_decision_type decision;
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1016
6709 395082a34cf8 Reopen cache file early enough in the transaction if it has changed. Timo Sirainen <tss@iki.fi> parents: 6703 diff changeset	1017 mail_cache_transaction_open_if_needed(ctx);
5966 731be87a2128 Make sure cache is opened when mail_cache_field_can_add() or Timo Sirainen <tss@iki.fi> parents: 5960 diff changeset	1018
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	1019 decision = mail_cache_field_get_decision(ctx->view->cache, field_idx);
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1020 if (decision == (MAIL_CACHE_DECISION_FORCED \| MAIL_CACHE_DECISION_NO))
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1021 return FALSE;
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1022
5735 5763c2548153 s/field/field_idx/ Timo Sirainen <tss@iki.fi> parents: 5734 diff changeset	1023 return mail_cache_field_exists(ctx->view, seq, field_idx) == 0;
4261 8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1024 }
8fd5dfa271ef Added mail_cache_field_want_add() and mail_cache_field_can_add() Timo Sirainen <timo.sirainen@movial.fi> parents: 3953 diff changeset	1025
2354 1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1026 static int mail_cache_link_unlocked(struct mail_cache *cache,
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1027 uint32_t old_offset, uint32_t new_offset)
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1028 {
1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1029 new_offset += offsetof(struct mail_cache_record, prev_offset);
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	1030 return mail_cache_write(cache, &old_offset, sizeof(old_offset),
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3277 diff changeset	1031 new_offset);
2354 1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1032 }
2315 22210d6cf238 Cache fixes. Lookups now look into transactions too. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	1033
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1034 int mail_cache_link(struct mail_cache *cache, uint32_t old_offset,
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1035 uint32_t new_offset)
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1036 {
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1037 const struct mail_cache_record *rec;
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1038
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1039 i_assert(cache->locked);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1040
2578 9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	1041 if (MAIL_CACHE_IS_UNUSABLE(cache))
9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	1042 return -1;
9c76f91ba94c If we detect cache corruption, try to avoid printing tons of "bad file Timo Sirainen <tss@iki.fi> parents: 2420 diff changeset	1043
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1044 /* this function is called for each added cache record (or cache
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1045 extension record update actually) with new_offset pointing to the
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1046 new record and old_offset pointing to the previous record.
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1047
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1048 we want to keep the old and new records linked so both old and new
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1049 cached data is found. normally they are already linked correctly.
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1050 the problem only comes when multiple processes are adding cache
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1051 records at the same time. we'd rather not lose those additions, so
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1052 force the linking order to be new_offset -> old_offset if it isn't
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1053 already. */
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1054 if (mail_cache_map(cache, new_offset, sizeof(*rec)) < 0)
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1055 return -1;
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1056 if (new_offset + sizeof(*rec) > cache->mmap_length) {
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1057 mail_cache_set_corrupted(cache,
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1058 "Cache record offset %u points outside file",
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1059 new_offset);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1060 return -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1061 }
7138 876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1062 rec = CACHE_RECORD(cache, new_offset);
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1063 if (rec->prev_offset == old_offset) {
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1064 /* link is already correct */
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1065 return 0;
876c7bca351c Link cache records together directly when writing the new records, instead Timo Sirainen <tss@iki.fi> parents: 7105 diff changeset	1066 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1067
2354 1556d9ba23dc Assert crashfix in certain situations when adding new cache records. Timo Sirainen <tss@iki.fi> parents: 2341 diff changeset	1068 if (mail_cache_link_unlocked(cache, old_offset, new_offset) < 0)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1069 return -1;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1070
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1071 cache->hdr_copy.continued_record_count++;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1072 cache->hdr_modified = TRUE;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1073 return 0;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1074 }
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1075
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1076 static int mail_cache_delete_real(struct mail_cache *cache, uint32_t offset)
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1077 {
5734 5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1078 const struct mail_cache_record *rec;
7312 6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1079 struct mail_cache_loop_track loop_track;
6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1080 int ret = 0;
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1081
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1082 i_assert(cache->locked);
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1083
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1084 /* we'll only update the deleted_space in header. we can't really
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1085 do any actual deleting as other processes might still be using
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1086 the data. also it's actually useful as some index views are still
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1087 able to ask cached data from messages that have already been
41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1088 expunged. */
7312 6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1089 memset(&loop_track, 0, sizeof(loop_track));
6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1090 while (offset != 0 &&
6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1091 (ret = mail_cache_get_record(cache, offset, &rec)) == 0) {
6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1092 if (mail_cache_track_loops(&loop_track, offset, rec->size)) {
3688 1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1093 mail_cache_set_corrupted(cache,
1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1094 "record list is circular");
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1095 return -1;
3688 1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1096 }
7312 6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1097
6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1098 cache->hdr_copy.deleted_space += rec->size;
6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1099 offset = rec->prev_offset;
3688 1e6af9a000e6 mail_cache_delete(): Make sure we don't get to infinite loop if cache Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	1100 }
7312 6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1101 return ret;
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1102 }
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1103
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1104 int mail_cache_delete(struct mail_cache *cache, uint32_t offset)
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1105 {
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1106 int ret;
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1107
7312 6f59434cf229 Optimized cache record loop tracking. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	1108 i_assert(cache->locked);
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7199 diff changeset	1109 T_BEGIN {
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6841 diff changeset	1110 ret = mail_cache_delete_real(cache, offset);
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7199 diff changeset	1111 } T_END;
2277 41e56f28d085 Cache updating is done now by first reserving space where to write, and then Timo Sirainen <tss@iki.fi> parents: 2276 diff changeset	1112 cache->hdr_modified = TRUE;
5734 5f0832288007 Cache code cleanups Timo Sirainen <tss@iki.fi> parents: 5733 diff changeset	1113 return ret;
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	1114 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-cache-transaction.c @ 9458:adee8cb3ff5d HEAD