dovecot/original-hg/dovecot-1.2: src/lib-index/mail-cache-compress.c annotate

annotate src/lib-index/mail-cache-compress.c @ 3627:c0e35566dd14 HEAD

Unlocking cache file can also corrupt it since it modifies it. Added a return value for it and checks to handle the failure. Fixes a crash in some error situations.

author	Timo Sirainen <tss@iki.fi>
date	Fri, 30 Sep 2005 21:46:30 +0300
parents	904849549eac
children	61f0e1fe9b55

rev	line source
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	1 /* Copyright (C) 2003-2004 Timo Sirainen */
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	2
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	3 #include "lib.h"
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	4 #include "buffer.h"
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	5 #include "ostream.h"
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2950 diff changeset	6 #include "file-dotlock.h"
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	7 #include "file-cache.h"
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	8 #include "file-set-size.h"
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	9 #include "mail-cache-private.h"
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	10
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	11 #include <sys/stat.h>
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	12
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	13 struct mail_cache_copy_context {
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	14 int new_msg;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	15 buffer_t buffer, field_seen;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	16 uint8_t field_seen_value;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	17 };
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	18
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	19 static void mail_cache_merge_bitmask(struct mail_cache cache, buffer_t buffer,
2619 ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	20 uint32_t field, const void *data,
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	21 size_t data_size)
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	22 {
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	23 void *buf_data;
2620 f4f793129402 fix Timo Sirainen <tss@iki.fi> parents: 2619 diff changeset	24 uint32_t buf_field;
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	25 unsigned int i, buf_data_size;
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	26 size_t pos, buf_size;
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	27
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	28 buf_data = buffer_get_modifyable_data(buffer, &buf_size);
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	29 for (pos = sizeof(struct mail_cache_record); pos < buf_size; ) {
2620 f4f793129402 fix Timo Sirainen <tss@iki.fi> parents: 2619 diff changeset	30 buf_field = ((uint32_t )PTR_OFFSET(buf_data, pos));
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	31 pos += sizeof(uint32_t);
2700 5b07b6fd9bf8 Added asserts Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	32 i_assert(buf_field < cache->fields_count);
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	33
2623 0157839e3c78 bitmask merging code was corrupting the cache file Timo Sirainen <tss@iki.fi> parents: 2620 diff changeset	34 buf_data_size = cache->fields[buf_field].field.field_size;
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	35 if (buf_data_size == (unsigned int)-1) {
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	36 buf_data_size =
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	37 ((uint32_t )PTR_OFFSET(buf_data, pos));
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	38 pos += sizeof(uint32_t);
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	39 }
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	40
2620 f4f793129402 fix Timo Sirainen <tss@iki.fi> parents: 2619 diff changeset	41 if (buf_field == field) {
2700 5b07b6fd9bf8 Added asserts Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	42 /* @UNSAFE: found it, do the merging */
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	43 unsigned char *dest = PTR_OFFSET(buf_data, pos);
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	44
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	45 i_assert(buf_data_size == data_size);
2700 5b07b6fd9bf8 Added asserts Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	46 i_assert(pos + buf_data_size <= buf_size);
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	47 for (i = 0; i < buf_data_size; i++)
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	48 dest[i] \|= ((const unsigned char*)data)[i];
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	49 break;
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	50 }
2950 30110ac226a7 mail_cache_merge_bitmask() was broken, cache compression could have crashed Timo Sirainen <tss@iki.fi> parents: 2949 diff changeset	51 pos += (buf_data_size + 3) & ~3;
2700 5b07b6fd9bf8 Added asserts Timo Sirainen <tss@iki.fi> parents: 2671 diff changeset	52 i_assert(pos <= buf_size);
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	53 }
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	54 }
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	55
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	56 static int
2619 ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	57 mail_cache_compress_callback(struct mail_cache_view *view, uint32_t field,
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	58 const void data, size_t data_size, void context)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	59 {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	60 struct mail_cache_copy_context *ctx = context;
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	61 struct mail_cache_field *cache_field;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	62 enum mail_cache_decision_type dec;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	63 uint8_t *field_seen;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	64 uint32_t size32;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	65
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	66 cache_field = &view->cache->fields[field].field;
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	67
2619 ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	68 field_seen = buffer_get_space_unsafe(ctx->field_seen, field, 1);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	69 if (*field_seen == ctx->field_seen_value) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	70 /* duplicate */
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	71 if (cache_field->type == MAIL_CACHE_FIELD_BITMASK) {
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	72 mail_cache_merge_bitmask(view->cache, ctx->buffer,
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	73 field, data, data_size);
87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	74 }
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	75 return 1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	76 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	77 *field_seen = ctx->field_seen_value;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	78
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	79 dec = cache_field->decision & ~MAIL_CACHE_DECISION_FORCED;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	80 if (ctx->new_msg) {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	81 if (dec == MAIL_CACHE_DECISION_NO)
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	82 return 1;
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	83 } else {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	84 if (dec != MAIL_CACHE_DECISION_YES)
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	85 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	86 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	87
2619 ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	88 buffer_append(ctx->buffer, &field, sizeof(field));
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	89
2555 87af1f39a69e Added support for bitmask type. Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	90 if (cache_field->field_size == (unsigned int)-1) {
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	91 size32 = (uint32_t)data_size;
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	92 buffer_append(ctx->buffer, &size32, sizeof(size32));
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	93 }
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	94
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	95 buffer_append(ctx->buffer, data, data_size);
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	96 if ((data_size & 3) != 0)
3617 904849549eac Removed null4 and replaced the few uses of it with buffer_append_zero(). Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	97 buffer_append_zero(ctx->buffer, 4 - (data_size & 3));
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	98 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	99 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	100
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	101 static uint32_t
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	102 get_next_file_seq(struct mail_cache cache, struct mail_index_view view)
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	103 {
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	104 const struct mail_index_ext *ext;
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	105 uint32_t file_seq;
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	106
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	107 ext = mail_index_view_get_ext(view, cache->ext_id);
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	108 file_seq = ext != NULL ? ext->reset_id + 1 : (uint32_t)ioloop_time;
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	109 return file_seq != 0 ? file_seq : 1;
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	110 }
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	111
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	112 static int
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	113 mail_cache_copy(struct mail_cache cache, struct mail_index_view view, int fd)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	114 {
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	115 struct mail_cache_copy_context ctx;
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	116 struct mail_cache_view *cache_view;
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	117 struct mail_index_transaction *t;
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	118 const struct mail_index_header *idx_hdr;
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	119 struct mail_cache_header hdr;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	120 struct mail_cache_record cache_rec;
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	121 struct ostream *output;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	122 buffer_t *buffer;
2619 ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	123 uint32_t i, message_count, seq, first_new_seq, old_offset;
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	124 uoff_t offset;
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	125
3551 65dc45982df9 Removed fcntl() text from some locking error messages, because if Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	126 /* get sequence of first message which doesn't need its temp fields
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	127 removed. */
2892 62d53b49110d Changed mail_index_get_header() to return the header as return value because Timo Sirainen <tss@iki.fi> parents: 2879 diff changeset	128 idx_hdr = mail_index_get_header(view);
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	129 if (idx_hdr->day_first_uid[7] == 0) {
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	130 first_new_seq = 1;
2904 642ba9cd3dac Removed view->messages_count, view->hdr.messages_count is enough. Also fixes Timo Sirainen <tss@iki.fi> parents: 2892 diff changeset	131 message_count = mail_index_view_get_messages_count(view);
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	132 } else {
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	133 if (mail_index_lookup_uid_range(view, idx_hdr->day_first_uid[7],
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	134 (uint32_t)-1, &first_new_seq,
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	135 &message_count) < 0)
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	136 return -1;
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	137 if (first_new_seq == 0)
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	138 first_new_seq = message_count+1;
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	139 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	140
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	141 cache_view = mail_cache_view_open(cache, view);
2877 3cb483d565a6 Allow specifying to transaction if it's external. Make mail saving Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	142 t = mail_index_transaction_begin(view, FALSE, TRUE);
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	143 output = o_stream_create_file(fd, default_pool, 0, FALSE);
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	144
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	145 memset(&hdr, 0, sizeof(hdr));
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	146 hdr.version = MAIL_CACHE_VERSION;
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	147 hdr.indexid = idx_hdr->indexid;
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	148 hdr.file_seq = get_next_file_seq(cache, view);
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	149 o_stream_send(output, &hdr, sizeof(hdr));
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	150
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	151 memset(&ctx, 0, sizeof(ctx));
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2700 diff changeset	152 ctx.buffer = buffer_create_dynamic(default_pool, 4096);
f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2700 diff changeset	153 ctx.field_seen = buffer_create_dynamic(default_pool, 64);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	154 ctx.field_seen_value = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	155
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	156 mail_index_ext_reset(t, cache->ext_id, hdr.file_seq);
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	157
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	158 for (seq = 1; seq <= message_count; seq++) {
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	159 ctx.new_msg = seq >= first_new_seq;
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	160 buffer_set_used_size(ctx.buffer, 0);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	161
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	162 if (++ctx.field_seen_value == 0) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	163 memset(buffer_get_modifyable_data(ctx.field_seen, NULL),
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	164 0, buffer_get_size(ctx.field_seen));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	165 ctx.field_seen_value++;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	166 }
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	167
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	168 memset(&cache_rec, 0, sizeof(cache_rec));
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	169 buffer_append(ctx.buffer, &cache_rec, sizeof(cache_rec));
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	170
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	171 (void)mail_cache_foreach(cache_view, seq,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	172 mail_cache_compress_callback, &ctx);
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	173
2299 ef0cd18d674c Fixed compression. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	174 cache_rec.size = buffer_get_used_size(ctx.buffer);
ef0cd18d674c Fixed compression. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	175 if (cache_rec.size == sizeof(cache_rec))
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	176 continue;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	177
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	178 mail_index_update_ext(t, seq, cache->ext_id, &output->offset,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	179 &old_offset);
2299 ef0cd18d674c Fixed compression. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	180
ef0cd18d674c Fixed compression. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	181 buffer_write(ctx.buffer, 0, &cache_rec, sizeof(cache_rec));
2853 512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	182 o_stream_send(output, ctx.buffer->data, cache_rec.size);
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	183 }
2619 ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	184
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	185 if (cache->fields_count != 0) {
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	186 hdr.field_header_offset =
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	187 mail_index_uint32_to_offset(output->offset);
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	188
2671 3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in Timo Sirainen <tss@iki.fi> parents: 2641 diff changeset	189 /* we wrote everything using our internal field ids. so we want
2619 ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	190 mail_cache_header_fields_get() to use them and ignore any
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	191 existing id mappings in the old cache file. */
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	192 cache->file_fields_count = 0;
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	193 for (i = 0; i < cache->fields_count; i++)
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	194 cache->field_file_map[i] = (uint32_t)-1;
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	195
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	196 t_push();
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2700 diff changeset	197 buffer = buffer_create_dynamic(pool_datastack_create(), 256);
2619 ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	198 mail_cache_header_fields_get(cache, buffer);
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	199 o_stream_send(output, buffer_get_data(buffer, NULL),
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	200 buffer_get_used_size(buffer));
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	201 t_pop();
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	202 }
ce8647d94bfc ID mapping fixes. Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	203
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	204 hdr.used_file_size = output->offset;
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	205 buffer_free(ctx.buffer);
2641 e357ebb3137b Memory leak fixes Timo Sirainen <tss@iki.fi> parents: 2623 diff changeset	206 buffer_free(ctx.field_seen);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	207
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	208 o_stream_seek(output, 0);
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	209 o_stream_send(output, &hdr, sizeof(hdr));
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	210
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	211 mail_cache_view_close(cache_view);
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	212
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	213 if (o_stream_flush(output) < 0) {
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	214 errno = output->stream_errno;
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	215 mail_cache_set_syscall_error(cache, "o_stream_flush()");
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	216 (void)mail_index_transaction_rollback(t);
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	217 o_stream_unref(output);
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	218 return -1;
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	219 }
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	220
2299 ef0cd18d674c Fixed compression. Timo Sirainen <tss@iki.fi> parents: 2298 diff changeset	221 if (hdr.used_file_size < MAIL_CACHE_INITIAL_SIZE) {
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	222 /* grow the file some more. doesn't matter if it fails */
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	223 (void)file_set_size(fd, MAIL_CACHE_INITIAL_SIZE);
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	224 }
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	225
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	226 o_stream_unref(output);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	227
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	228 if (fdatasync(fd) < 0) {
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	229 mail_cache_set_syscall_error(cache, "fdatasync()");
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	230 (void)mail_index_transaction_rollback(t);
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	231 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	232 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	233
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	234 return mail_index_transaction_commit(t, &seq, &offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	235 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	236
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	237 static int mail_cache_compress_locked(struct mail_cache *cache,
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	238 struct mail_index_view *view)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	239 {
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2950 diff changeset	240 struct dotlock *dotlock;
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	241 mode_t old_mask;
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	242 int fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	243
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	244 /* get the latest info on fields */
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	245 if (mail_cache_header_fields_read(cache) < 0)
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	246 return -1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: 2299 diff changeset	247
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	248 #ifdef DEBUG
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	249 i_warning("Compressing cache file %s", cache->filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	250 #endif
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	251
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	252 old_mask = umask(cache->index->mode ^ 0666);
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2950 diff changeset	253 fd = file_dotlock_open(&cache->dotlock_settings, cache->filepath,
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2950 diff changeset	254 0, &dotlock);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	255 umask(old_mask);
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	256
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	257 if (fd == -1) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	258 mail_cache_set_syscall_error(cache, "file_dotlock_open()");
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	259 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	260 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	261
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	262 if (cache->index->gid != (gid_t)-1 &&
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	263 fchown(fd, (uid_t)-1, cache->index->gid) < 0) {
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	264 mail_cache_set_syscall_error(cache, "fchown()");
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2950 diff changeset	265 file_dotlock_delete(&dotlock);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	266 return -1;
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	267 }
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	268
2298 5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but Timo Sirainen <tss@iki.fi> parents: 2282 diff changeset	269 // FIXME: check that cache file wasn't just recreated
2276 5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code - Timo Sirainen <tss@iki.fi> parents: 2275 diff changeset	270
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	271 if (mail_cache_copy(cache, view, fd) < 0) {
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2950 diff changeset	272 (void)file_dotlock_delete(&dotlock);
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	273 return -1;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	274 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	275
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2950 diff changeset	276 if (file_dotlock_replace(&dotlock,
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2950 diff changeset	277 DOTLOCK_REPLACE_FLAG_DONT_CLOSE_FD) < 0) {
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	278 mail_cache_set_syscall_error(cache,
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	279 "file_dotlock_replace()");
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	280 (void)close(fd);
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	281 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	282 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	283
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	284 mail_cache_file_close(cache);
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	285 cache->fd = fd;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	286
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	287 if (cache->file_cache != NULL)
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	288 file_cache_set_fd(cache->file_cache, cache->fd);
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	289
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	290 if (mail_cache_map(cache, 0, 0) < 0)
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	291 return -1;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	292 if (mail_cache_header_fields_read(cache) < 0)
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	293 return -1;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	294
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	295 cache->need_compress = FALSE;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	296 return 0;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	297 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	298
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	299 int mail_cache_compress(struct mail_cache cache, struct mail_index_view view)
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	300 {
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	301 int ret;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	302
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3106 diff changeset	303 if (MAIL_INDEX_IS_IN_MEMORY(cache->index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3106 diff changeset	304 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3106 diff changeset	305
2949 05ae72649fd0 Cache compression didn't work with lock_method = dotlock. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	306 if (cache->index->lock_method == MAIL_INDEX_LOCK_DOTLOCK) {
05ae72649fd0 Cache compression didn't work with lock_method = dotlock. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	307 /* we're using dotlocking, cache file creation itself creates
05ae72649fd0 Cache compression didn't work with lock_method = dotlock. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	308 the dotlock file we need. */
05ae72649fd0 Cache compression didn't work with lock_method = dotlock. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	309 return mail_cache_compress_locked(cache, view);
05ae72649fd0 Cache compression didn't work with lock_method = dotlock. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	310 }
05ae72649fd0 Cache compression didn't work with lock_method = dotlock. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	311
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	312 switch (mail_cache_lock(cache)) {
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	313 case -1:
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	314 return -1;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	315 case 0:
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	316 /* couldn't lock, either it's broken or doesn't exist.
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	317 just start creating it. */
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	318 return mail_cache_compress_locked(cache, view);
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	319 default:
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	320 /* locking succeeded. */
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	321 ret = mail_cache_compress_locked(cache, view);
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3617 diff changeset	322 if (mail_cache_unlock(cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3617 diff changeset	323 ret = -1;
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	324 return ret;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2904 diff changeset	325 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	326 }
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	327
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	328 int mail_cache_need_compress(struct mail_cache *cache)
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	329 {
2275 c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened. Timo Sirainen <tss@iki.fi> parents: 2257 diff changeset	330 return cache->need_compress;
2247 2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested. Timo Sirainen <tss@iki.fi> parents: 2200 diff changeset	331 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-cache-compress.c @ 3627:c0e35566dd14 HEAD