dovecot/original-hg/dovecot-1.2: src/lib-index/mail-cache-fields.c annotate

annotate src/lib-index/mail-cache-fields.c @ 3999:f1e29736e60d HEAD

Don't register cache fields twice. Fixes "duplicate field in header" errors.

author	Timo Sirainen <tss@iki.fi>
date	Wed, 08 Feb 2006 19:52:45 +0200
parents	c0e35566dd14
children	fcfd44f56b04

rev	line source
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	1 /* Copyright (C) 2004 Timo Sirainen */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	2
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	4 #include "buffer.h"
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "hash.h"
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	6 #include "file-cache.h"
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	7 #include "write-full.h"
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	8 #include "mail-cache-private.h"
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	9
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	10 #include <stddef.h>
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	11
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	12 #define CACHE_HDR_PREFETCH 1024
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	13
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	14 void mail_cache_register_fields(struct mail_cache *cache,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	15 struct mail_cache_field *fields,
3278 df372eecc5d0 Several size_t -> unsigned int changes. Try to keep "unsigned int" for Timo Sirainen <tss@iki.fi> parents: 2968 diff changeset	16 unsigned int fields_count)
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	17 {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	18 void orig_key, orig_value;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	19 unsigned int new_idx;
3999 f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	20 unsigned int i, j;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	21
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	22 new_idx = cache->fields_count;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	23 for (i = 0; i < fields_count; i++) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	24 if (hash_lookup_full(cache->field_name_hash, fields[i].name,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	25 &orig_key, &orig_value)) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	26 fields[i].idx =
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	27 POINTER_CAST_TO(orig_value, unsigned int);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	28 continue;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	29 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	30
3999 f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	31 /* check if the same header is being registered in the
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	32 same field array */
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	33 for (j = 0; j < i; j++) {
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	34 if (strcasecmp(fields[i].name, fields[j].name) == 0) {
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	35 fields[i].idx = fields[j].idx;
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	36 break;
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	37 }
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	38 }
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	39
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	40 if (j == i)
f1e29736e60d Don't register cache fields twice. Fixes "duplicate field in header" errors. Timo Sirainen <tss@iki.fi> parents: 3627 diff changeset	41 fields[i].idx = new_idx++;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	42 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	43
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	44 if (new_idx == cache->fields_count)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	45 return;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	46
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	47 /* @UNSAFE */
2362 f2acbe281ac1 Allocate cache->fields and cache->fields_file_map with malloc rather than Timo Sirainen <tss@iki.fi> parents: 2342 diff changeset	48 cache->fields = i_realloc(cache->fields,
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	49 cache->fields_count * sizeof(*cache->fields),
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	50 new_idx * sizeof(*cache->fields));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	51 cache->field_file_map =
2362 f2acbe281ac1 Allocate cache->fields and cache->fields_file_map with malloc rather than Timo Sirainen <tss@iki.fi> parents: 2342 diff changeset	52 i_realloc(cache->field_file_map,
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	53 cache->fields_count * sizeof(*cache->field_file_map),
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	54 new_idx * sizeof(*cache->field_file_map));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	55
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	56 for (i = 0; i < fields_count; i++) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	57 unsigned int idx = fields[i].idx;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	58
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	59 if (idx < cache->fields_count)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	60 continue;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	61
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	62 /* new index - save it */
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	63 cache->fields[idx].field = fields[i];
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	64 cache->fields[idx].field.name =
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	65 p_strdup(cache->field_pool, fields[i].name);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	66 cache->field_file_map[idx] = (uint32_t)-1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	67
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	68 switch (cache->fields[idx].field.type) {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	69 case MAIL_CACHE_FIELD_FIXED_SIZE:
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	70 case MAIL_CACHE_FIELD_BITMASK:
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	71 break;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	72 case MAIL_CACHE_FIELD_VARIABLE_SIZE:
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	73 case MAIL_CACHE_FIELD_STRING:
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	74 case MAIL_CACHE_FIELD_HEADER:
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	75 cache->fields[idx].field.field_size = (unsigned int)-1;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	76 break;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	77 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	78
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	79 hash_insert(cache->field_name_hash,
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	80 (char *)cache->fields[idx].field.name,
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	81 POINTER_CAST(idx));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	82 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	83 cache->fields_count = new_idx;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	84 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	85
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	86 unsigned int
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	87 mail_cache_register_lookup(struct mail_cache cache, const char name)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	88 {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	89 void orig_key, orig_value;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	90
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	91 if (hash_lookup_full(cache->field_name_hash, name,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	92 &orig_key, &orig_value))
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	93 return POINTER_CAST_TO(orig_value, unsigned int);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	94 else
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	95 return (unsigned int)-1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	96 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	97
3461 b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	98 const struct mail_cache_field *
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	99 mail_cache_register_get_list(struct mail_cache *cache, pool_t pool,
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	100 unsigned int *count_r)
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	101 {
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	102 struct mail_cache_field *list;
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	103 unsigned int i;
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	104
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	105 list = p_new(pool, struct mail_cache_field, cache->fields_count);
3494 7cab5dc6a0cd mail_cache_register_get_list(): Allocate the returned strings also from Timo Sirainen <tss@iki.fi> parents: 3461 diff changeset	106 for (i = 0; i < cache->fields_count; i++) {
3461 b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	107 list[i] = cache->fields[i].field;
3494 7cab5dc6a0cd mail_cache_register_get_list(): Allocate the returned strings also from Timo Sirainen <tss@iki.fi> parents: 3461 diff changeset	108 list[i].name = p_strdup(pool, list[i].name);
7cab5dc6a0cd mail_cache_register_get_list(): Allocate the returned strings also from Timo Sirainen <tss@iki.fi> parents: 3461 diff changeset	109 }
3461 b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	110
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	111 *count_r = cache->fields_count;
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	112 return list;
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	113 }
b7ce2532250a Added mail_cache_register_get_list() to get a list of all registered fields Timo Sirainen <tss@iki.fi> parents: 3375 diff changeset	114
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	115 static int mail_cache_header_fields_get_offset(struct mail_cache *cache,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	116 uint32_t *offset_r)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	117 {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	118 const struct mail_cache_header_fields *field_hdr;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	119 uint32_t offset, next_offset;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	120
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	121 if (MAIL_CACHE_IS_UNUSABLE(cache)) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	122 *offset_r = 0;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	123 return 0;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	124 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	125
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	126 /* find the latest header */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	127 offset = 0;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	128 next_offset =
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2362 diff changeset	129 mail_index_offset_to_uint32(cache->hdr->field_header_offset);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	130 while (next_offset != 0) {
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	131 if (next_offset == offset) {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	132 mail_cache_set_corrupted(cache,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	133 "next_offset in field header loops");
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	134 return -1;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	135 }
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	136 offset = next_offset;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	137
2867 d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	138 if (cache->file_cache != NULL) {
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	139 /* we can't trust that the cached data is valid */
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	140 file_cache_invalidate(cache->file_cache, offset,
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	141 sizeof(*field_hdr) +
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	142 CACHE_HDR_PREFETCH);
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	143 }
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	144 if (mail_cache_map(cache, offset,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	145 sizeof(*field_hdr) + CACHE_HDR_PREFETCH) < 0)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	146 return -1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	147
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	148 field_hdr = CONST_PTR_OFFSET(cache->data, offset);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	149 next_offset =
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2362 diff changeset	150 mail_index_offset_to_uint32(field_hdr->next_offset);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	151 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	152
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	153 *offset_r = offset;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	154 return 0;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	155 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	156
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	157 int mail_cache_header_fields_read(struct mail_cache *cache)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	158 {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	159 const struct mail_cache_header_fields *field_hdr = NULL;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	160 struct mail_cache_field field;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	161 const uint32_t last_used, sizes;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	162 const uint8_t types, decisions;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	163 const char p, names, *end;
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	164 void orig_key, orig_value;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	165 uint32_t offset, i;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	166
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	167 if (mail_cache_header_fields_get_offset(cache, &offset) < 0)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	168 return -1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	169
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	170 if (offset == 0) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	171 /* no fields - the file is empty */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	172 return 0;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	173 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	174
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	175 field_hdr = CONST_PTR_OFFSET(cache->data, offset);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	176 if (offset + field_hdr->size > cache->mmap_length) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	177 mail_cache_set_corrupted(cache,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	178 "field header points outside file");
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	179 return -1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	180 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	181
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	182 /* check the fixed size of the header. name[] has to be checked
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	183 separately */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	184 if (field_hdr->size < sizeof(*field_hdr) +
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	185 field_hdr->fields_count * (sizeof(uint32_t)*2 + 1 + 2)) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	186 mail_cache_set_corrupted(cache, "invalid field header size");
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	187 return -1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	188 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	189
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	190 if (field_hdr->size > sizeof(*field_hdr) + CACHE_HDR_PREFETCH) {
2867 d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	191 if (cache->file_cache != NULL) {
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	192 /* we can't trust that the cached data is valid */
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	193 file_cache_invalidate(cache->file_cache, offset,
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	194 sizeof(*field_hdr) +
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	195 CACHE_HDR_PREFETCH);
d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	196 }
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	197 if (mail_cache_map(cache, offset, field_hdr->size) < 0)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	198 return -1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	199 }
2866 bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations. Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	200 field_hdr = CONST_PTR_OFFSET(cache->data, offset);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	201
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	202 cache->file_field_map =
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	203 i_realloc(cache->file_field_map,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	204 cache->file_fields_count * sizeof(unsigned int),
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	205 field_hdr->fields_count * sizeof(unsigned int));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	206 cache->file_fields_count = field_hdr->fields_count;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	207
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	208 last_used = CONST_PTR_OFFSET(field_hdr, MAIL_CACHE_FIELD_LAST_USED());
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	209 sizes = CONST_PTR_OFFSET(field_hdr,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	210 MAIL_CACHE_FIELD_SIZE(field_hdr->fields_count));
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	211 types = CONST_PTR_OFFSET(field_hdr,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	212 MAIL_CACHE_FIELD_TYPE(field_hdr->fields_count));
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	213 decisions = CONST_PTR_OFFSET(field_hdr,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	214 MAIL_CACHE_FIELD_DECISION(field_hdr->fields_count));
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	215 names = CONST_PTR_OFFSET(field_hdr,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	216 MAIL_CACHE_FIELD_NAMES(field_hdr->fields_count));
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	217 end = CONST_PTR_OFFSET(field_hdr, field_hdr->size);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	218
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	219 /* clear the old mapping */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	220 for (i = 0; i < cache->fields_count; i++)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	221 cache->field_file_map[i] = (uint32_t)-1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	222
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	223 memset(&field, 0, sizeof(field));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	224 for (i = 0; i < field_hdr->fields_count; i++) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	225 for (p = names; p != end && *p != '\0'; p++) ;
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	226 if (p == end \|\| *names == '\0') {
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	227 mail_cache_set_corrupted(cache,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	228 "field header names corrupted");
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	229 return -1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	230 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	231
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	232 if (hash_lookup_full(cache->field_name_hash, names,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	233 &orig_key, &orig_value)) {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	234 /* already exists, see if decision can be updated */
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	235 field.idx = POINTER_CAST_TO(orig_value, unsigned int);
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	236 if (!cache->fields[field.idx].decision_dirty) {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	237 cache->fields[field.idx].field.decision =
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	238 decisions[i];
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	239 }
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	240 if (cache->fields[field.idx].field.type != types[i]) {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	241 mail_cache_set_corrupted(cache,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	242 "registered field type changed");
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	243 return -1;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	244 }
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	245 } else {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	246 field.name = names;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	247 field.type = types[i];
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	248 field.field_size = sizes[i];
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	249 field.decision = decisions[i];
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	250 mail_cache_register_fields(cache, &field, 1);
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	251 }
2968 2ab037df6cf3 Check for duplicate fields in header. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	252 if (cache->field_file_map[field.idx] != (uint32_t)-1) {
2ab037df6cf3 Check for duplicate fields in header. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	253 mail_cache_set_corrupted(cache,
2ab037df6cf3 Check for duplicate fields in header. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	254 "Duplicated field in header: %s", names);
2ab037df6cf3 Check for duplicate fields in header. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	255 return -1;
2ab037df6cf3 Check for duplicate fields in header. Timo Sirainen <tss@iki.fi> parents: 2929 diff changeset	256 }
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	257 cache->field_file_map[field.idx] = i;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	258 cache->file_field_map[i] = field.idx;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	259
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	260 /* update last_used if it's newer than ours */
2342 fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	261 if (last_used[i] > cache->fields[field.idx].last_used)
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	262 cache->fields[field.idx].last_used = last_used[i];
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	263
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	264 names = p + 1;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	265 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	266 return 0;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	267 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	268
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	269 static void copy_to_buf(struct mail_cache cache, buffer_t dest,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	270 size_t offset, size_t size)
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	271 {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	272 const void *data;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	273 unsigned int i, field;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	274
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	275 for (i = 0; i < cache->file_fields_count; i++) {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	276 field = cache->file_field_map[i];
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	277 data = CONST_PTR_OFFSET(&cache->fields[field], offset);
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	278 buffer_append(dest, data, size);
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	279 }
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	280 for (i = 0; i < cache->fields_count; i++) {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	281 if (cache->field_file_map[i] != (uint32_t)-1)
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	282 continue;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	283 data = CONST_PTR_OFFSET(&cache->fields[i], offset);
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	284 buffer_append(dest, data, size);
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	285 }
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	286 }
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	287
2342 fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	288 static void copy_to_buf_byte(struct mail_cache cache, buffer_t dest,
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	289 size_t offset)
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	290 {
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	291 const int *data;
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	292 unsigned int i, field;
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	293 uint8_t byte;
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	294
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	295 for (i = 0; i < cache->file_fields_count; i++) {
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	296 field = cache->file_field_map[i];
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	297 data = CONST_PTR_OFFSET(&cache->fields[field], offset);
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	298 byte = (uint8_t)*data;
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	299 buffer_append(dest, &byte, 1);
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	300 }
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	301 for (i = 0; i < cache->fields_count; i++) {
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	302 if (cache->field_file_map[i] != (uint32_t)-1)
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	303 continue;
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	304 data = CONST_PTR_OFFSET(&cache->fields[i], offset);
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	305 byte = (uint8_t)*data;
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	306 buffer_append(dest, &byte, 1);
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	307 }
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	308 }
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	309
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	310 static int mail_cache_header_fields_update_locked(struct mail_cache *cache)
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	311 {
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	312 buffer_t *buffer;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	313 uint32_t i, offset;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	314 int ret = 0;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	315
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	316 if (mail_cache_header_fields_read(cache) < 0 \|\|
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	317 mail_cache_header_fields_get_offset(cache, &offset) < 0)
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	318 return -1;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	319
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	320 t_push();
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2604 diff changeset	321 buffer = buffer_create_dynamic(pool_datastack_create(), 256);
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	322
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	323 copy_to_buf(cache, buffer,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	324 offsetof(struct mail_cache_field_private, last_used),
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	325 sizeof(uint32_t));
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3278 diff changeset	326 ret = mail_cache_write(cache, buffer->data,
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3278 diff changeset	327 sizeof(uint32_t) * cache->file_fields_count,
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3278 diff changeset	328 offset + MAIL_CACHE_FIELD_LAST_USED());
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	329 if (ret == 0) {
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	330 buffer_set_used_size(buffer, 0);
2342 fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	331 copy_to_buf_byte(cache, buffer,
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	332 offsetof(struct mail_cache_field, decision));
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	333
3375 9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with Timo Sirainen <tss@iki.fi> parents: 3278 diff changeset	334 ret = mail_cache_write(cache, buffer->data,
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	335 sizeof(uint8_t) * cache->file_fields_count, offset +
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	336 MAIL_CACHE_FIELD_DECISION(cache->file_fields_count));
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	337
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	338 if (ret == 0) {
2592 5edda9783458 minor fix Timo Sirainen <tss@iki.fi> parents: 2407 diff changeset	339 for (i = 0; i < cache->file_fields_count; i++)
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	340 cache->fields[i].decision_dirty = FALSE;
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	341 }
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	342 }
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	343 t_pop();
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	344
2867 d416b6d0a7ee More correct file cache invalidating. Timo Sirainen <tss@iki.fi> parents: 2866 diff changeset	345 if (ret == 0)
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	346 cache->field_header_write_pending = FALSE;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	347
2929 ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	348 return ret;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	349 }
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	350
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	351 int mail_cache_header_fields_update(struct mail_cache *cache)
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	352 {
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	353 int ret;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	354
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	355 if (cache->locked)
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	356 return mail_cache_header_fields_update_locked(cache);
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	357
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	358 if (mail_cache_lock(cache) <= 0)
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	359 return -1;
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	360
ba9062032877 Locking fixes and cleanups Timo Sirainen <tss@iki.fi> parents: 2867 diff changeset	361 ret = mail_cache_header_fields_update_locked(cache);
3627 c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3617 diff changeset	362 if (mail_cache_unlock(cache) < 0)
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a Timo Sirainen <tss@iki.fi> parents: 3617 diff changeset	363 ret = -1;
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	364 return ret;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	365 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	366
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	367 void mail_cache_header_fields_get(struct mail_cache cache, buffer_t dest)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	368 {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	369 struct mail_cache_header_fields hdr;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	370 unsigned int field;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	371 const char *name;
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	372 uint32_t i;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	373
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	374 memset(&hdr, 0, sizeof(hdr));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	375 hdr.fields_count = cache->fields_count;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	376 buffer_append(dest, &hdr, sizeof(hdr));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	377
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	378 /* we have to keep the field order for the existing fields. */
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	379 copy_to_buf(cache, dest,
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	380 offsetof(struct mail_cache_field_private, last_used),
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	381 sizeof(uint32_t));
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	382 copy_to_buf(cache, dest, offsetof(struct mail_cache_field, field_size),
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	383 sizeof(uint32_t));
2342 fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	384 copy_to_buf_byte(cache, dest, offsetof(struct mail_cache_field, type));
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	385 copy_to_buf_byte(cache, dest,
fa529dd77176 Fixes for big endian systems. Timo Sirainen <tss@iki.fi> parents: 2339 diff changeset	386 offsetof(struct mail_cache_field, decision));
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	387
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	388 i_assert(buffer_get_used_size(dest) == sizeof(hdr) +
406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	389 (sizeof(uint32_t)2 + 2) hdr.fields_count);
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	390
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	391 for (i = 0; i < cache->file_fields_count; i++) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	392 field = cache->file_field_map[i];
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	393 name = cache->fields[field].field.name;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	394 buffer_append(dest, name, strlen(name)+1);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	395 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	396 for (i = 0; i < cache->fields_count; i++) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	397 if (cache->field_file_map[i] != (uint32_t)-1)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	398 continue;
2339 406692edc12d Cache fixes. Decisions are saved again. Timo Sirainen <tss@iki.fi> parents: 2327 diff changeset	399 name = cache->fields[i].field.name;
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	400 buffer_append(dest, name, strlen(name)+1);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	401 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	402
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	403 hdr.size = buffer_get_used_size(dest);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	404 buffer_write(dest, 0, &hdr, sizeof(hdr));
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	405
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	406 if ((hdr.size & 3) != 0)
3617 904849549eac Removed null4 and replaced the few uses of it with buffer_append_zero(). Timo Sirainen <tss@iki.fi> parents: 3494 diff changeset	407 buffer_append_zero(dest, 4 - (hdr.size & 3));
2327 7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	408 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	409
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	410 int mail_cache_header_fields_get_next_offset(struct mail_cache *cache,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	411 uint32_t *offset_r)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	412 {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	413 if (mail_cache_header_fields_get_offset(cache, offset_r) < 0)
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	414 return -1;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	415
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	416 if (*offset_r == 0) {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	417 *offset_r = offsetof(struct mail_cache_header,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	418 field_header_offset);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	419 } else {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	420 *offset_r += offsetof(struct mail_cache_header_fields,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	421 next_offset);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	422 }
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	423 return 0;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so Timo Sirainen <tss@iki.fi> parents: diff changeset	424 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-cache-fields.c @ 3999:f1e29736e60d HEAD