Mercurial > dovecot > core-2.2

annotate src/lib-index/mail-cache-private.h @ 2999:1d328531c466 HEAD

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
mail_cache_decision_*() prototypes were wrong.
author Timo Sirainen <tss@iki.fi>
date Sun, 19 Dec 2004 10:19:00 +0200
parents 89ab5c72430d
children 9c4aa309dbac
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1 #ifndef __MAIL_CACHE_PRIVATE_H
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
2 #define __MAIL_CACHE_PRIVATE_H
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
3
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
4 #include "mail-index-private.h"
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
5 #include "mail-cache.h"
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
6
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
7 #define MAIL_CACHE_VERSION 1
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
8
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
9 /* Never compress the file if it's smaller than this */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
10 #define COMPRESS_MIN_SIZE (1024*50)
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
11
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
12 /* Don't bother remembering holes smaller than this */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
13 #define MAIL_CACHE_MIN_HOLE_SIZE 1024
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
14
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
15 /* Compress the file when deleted space reaches n% of total size */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
16 #define COMPRESS_PERCENTAGE 20
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
17
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
18 /* Compress the file when n% of rows contain continued rows.
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
19 200% means that there's 2 continued rows per record. */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
20 #define COMPRESS_CONTINUED_PERCENTAGE 200
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
21
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
22 /* Initial size for the file */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
23 #define MAIL_CACHE_INITIAL_SIZE (sizeof(struct mail_cache_header) + 10240)
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
24
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
25 /* When more space is needed, grow the file n% larger than the previous size */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
26 #define MAIL_CACHE_GROW_PERCENTAGE 10
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
27
2341
b3c0ed666757 Limit how much a single transaction can reserve space
Timo Sirainen <tss@iki.fi>
parents: 2339
diff changeset
28 /* When allocating space for transactions, don't use blocks larger than this. */
b3c0ed666757 Limit how much a single transaction can reserve space
Timo Sirainen <tss@iki.fi>
parents: 2339
diff changeset
29 #define MAIL_CACHE_MAX_RESERVED_BLOCK_SIZE (1024*512)
b3c0ed666757 Limit how much a single transaction can reserve space
Timo Sirainen <tss@iki.fi>
parents: 2339
diff changeset
30
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
31 #define MAIL_CACHE_LOCK_TIMEOUT 120
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
32 #define MAIL_CACHE_LOCK_CHANGE_TIMEOUT 60
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
33 #define MAIL_CACHE_LOCK_IMMEDIATE_TIMEOUT (5*60)
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
34
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
35 #define CACHE_RECORD(cache, offset) \
2866
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents: 2853
diff changeset
36 ((const struct mail_cache_record *) \
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents: 2853
diff changeset
37 ((const char *) (cache)->data + offset))
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
38
2275
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
39 #define MAIL_CACHE_IS_UNUSABLE(cache) \
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
40 ((cache)->hdr == NULL)
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
41
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
42 struct mail_cache_header {
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
43 /* version is increased only when you can't have backwards
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
44 compatibility. */
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
45 uint8_t version;
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
46 uint8_t unused[3];
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
47
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
48 uint32_t indexid;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
49 uint32_t file_seq;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
50
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
51 uint32_t continued_record_count;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
52
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
53 uint32_t hole_offset;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
54 uint32_t used_file_size;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
55 uint32_t deleted_space;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
56
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
57 uint32_t field_header_offset;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
58 };
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
59
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
60 struct mail_cache_header_fields {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
61 uint32_t next_offset;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
62 uint32_t size;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
63 uint32_t fields_count;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
64
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
65 #if 0
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
66 /* last time the field was accessed. not updated more often than
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
67 once a day. */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
68 uint32_t last_used[fields_count];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
69 /* (uint32_t)-1 for variable sized fields */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
70 uint32_t size[fields_count];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
71 /* enum mail_cache_field_type */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
72 uint8_t type[fields_count];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
73 /* enum mail_cache_decision_type */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
74 uint8_t decision[fields_count];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
75 /* NUL-separated list of field names */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
76 char name[fields_count][];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
77 #endif
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
78 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
79
2339
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
80 #define MAIL_CACHE_FIELD_LAST_USED() \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
81 (sizeof(uint32_t) * 3)
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
82 #define MAIL_CACHE_FIELD_SIZE(count) \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
83 (MAIL_CACHE_FIELD_LAST_USED() + sizeof(uint32_t) * (count))
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
84 #define MAIL_CACHE_FIELD_TYPE(count) \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
85 (MAIL_CACHE_FIELD_SIZE(count) + sizeof(uint32_t) * (count))
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
86 #define MAIL_CACHE_FIELD_DECISION(count) \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
87 (MAIL_CACHE_FIELD_TYPE(count) + sizeof(uint8_t) * (count))
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
88 #define MAIL_CACHE_FIELD_NAMES(count) \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
89 (MAIL_CACHE_FIELD_DECISION(count) + sizeof(uint8_t) * (count))
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
90
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
91 struct mail_cache_record {
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
92 uint32_t prev_offset;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
93 uint32_t size; /* full record size, including this header */
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
94 /* array of { uint32_t field; [ uint32_t size; ] { .. } } */
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
95 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
96
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
97 struct mail_cache_hole_header {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
98 uint32_t next_offset; /* 0 if no holes left */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
99 uint32_t size; /* including this header */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
100
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
101 /* make sure we notice if we're treating hole as mail_cache_record.
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
102 magic is a large number so if it's treated as size field, it'll
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
103 point outside the file */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
104 #define MAIL_CACHE_HOLE_HEADER_MAGIC 0xffeedeff
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
105 uint32_t magic;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
106 };
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
107
2339
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
108 struct mail_cache_field_private {
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
109 struct mail_cache_field field;
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
110
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
111 uint32_t uid_highwater;
2342
fa529dd77176 Fixes for big endian systems.
Timo Sirainen <tss@iki.fi>
parents: 2341
diff changeset
112 uint32_t last_used;
2339
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
113
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
114 unsigned int decision_dirty:1;
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
115 };
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
116
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
117 struct mail_cache {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
118 struct mail_index *index;
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
119 uint32_t ext_id;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
120
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
121 char *filepath;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
122 int fd;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
123
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
124 void *mmap_base;
2866
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents: 2853
diff changeset
125 const void *data;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
126 size_t mmap_length;
2866
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents: 2853
diff changeset
127 struct file_cache *file_cache;
2941
89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file.
Timo Sirainen <tss@iki.fi>
parents: 2934
diff changeset
128 struct dotlock dotlock;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
129
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
130 const struct mail_cache_header *hdr;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
131 struct mail_cache_header hdr_copy;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
132
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
133 pool_t field_pool;
2339
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
134 struct mail_cache_field_private *fields;
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
135 uint32_t *field_file_map;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
136 unsigned int fields_count;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
137 struct hash_table *field_name_hash; /* name -> idx */
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
138
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
139 unsigned int *file_field_map;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
140 unsigned int file_fields_count;
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
141
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
142 unsigned int locked:1;
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
143 unsigned int need_compress:1;
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
144 unsigned int hdr_modified:1;
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
145 unsigned int field_header_write_pending:1;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
146 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
147
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
148 struct mail_cache_view {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
149 struct mail_cache *cache;
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
150 struct mail_index_view *view, *trans_view;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
151
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
152 struct mail_cache_transaction_ctx *transaction;
2315
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
153 uint32_t trans_seq1, trans_seq2;
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
154
2632
ec5601f71ba0 Fix for circular record list detection, we can't use data stack for buffer.
Timo Sirainen <tss@iki.fi>
parents: 2619
diff changeset
155 buffer_t *offsets_buf; /* temporary buffer, just to avoid mallocs */
ec5601f71ba0 Fix for circular record list detection, we can't use data stack for buffer.
Timo Sirainen <tss@iki.fi>
parents: 2619
diff changeset
156
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
157 /* if cached_exists_buf[field] == cached_exists_value, it's cached.
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
158 this allows us to avoid constantly clearing the whole buffer.
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
159 it needs to be cleared only when cached_exists_value is wrapped. */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
160 buffer_t *cached_exists_buf;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
161 uint8_t cached_exists_value;
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
162 uint32_t cached_exists_seq;
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
163 uint32_t cached_offset, cached_offset_seq;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
164 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
165
2315
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
166 typedef int mail_cache_foreach_callback_t(struct mail_cache_view *view,
2619
ce8647d94bfc ID mapping fixes.
Timo Sirainen <tss@iki.fi>
parents: 2407
diff changeset
167 uint32_t field,
2315
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
168 const void *data, size_t data_size,
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
169 void *context);
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
170
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
171 /* Explicitly lock the cache file. Returns -1 if error, 1 if ok, 0 if we
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
172 couldn't lock */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
173 int mail_cache_lock(struct mail_cache *cache);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
174 void mail_cache_unlock(struct mail_cache *cache);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
175
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
176 int mail_cache_header_fields_read(struct mail_cache *cache);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
177 int mail_cache_header_fields_update(struct mail_cache *cache);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
178 void mail_cache_header_fields_get(struct mail_cache *cache, buffer_t *dest);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
179 int mail_cache_header_fields_get_next_offset(struct mail_cache *cache,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
180 uint32_t *offset_r);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
181
2917
e657c7874962 Crashfixes for reading corrupted cache files.
Timo Sirainen <tss@iki.fi>
parents: 2866
diff changeset
182 int mail_cache_get_record(struct mail_cache *cache, uint32_t offset,
e657c7874962 Crashfixes for reading corrupted cache files.
Timo Sirainen <tss@iki.fi>
parents: 2866
diff changeset
183 const struct mail_cache_record **rec_r);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
184
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
185 int mail_cache_foreach(struct mail_cache_view *view, uint32_t seq,
2315
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
186 mail_cache_foreach_callback_t *callback, void *context);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
187
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
188 int mail_cache_transaction_commit(struct mail_cache_transaction_ctx *ctx);
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
189 void mail_cache_transaction_rollback(struct mail_cache_transaction_ctx *ctx);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
190
2275
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
191 int mail_cache_map(struct mail_cache *cache, size_t offset, size_t size);
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
192 void mail_cache_file_close(struct mail_cache *cache);
2275
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
193 int mail_cache_reopen(struct mail_cache *cache);
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
194
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
195 /* Update new_offset's prev_offset field to old_offset. */
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
196 int mail_cache_link(struct mail_cache *cache, uint32_t old_offset,
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
197 uint32_t new_offset);
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
198 /* Mark record in given offset to be deleted. */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
199 int mail_cache_delete(struct mail_cache *cache, uint32_t offset);
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
200
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
201 void mail_cache_decision_lookup(struct mail_cache_view *view, uint32_t seq,
2999
1d328531c466 mail_cache_decision_*() prototypes were wrong.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
202 unsigned int field);
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
203 void mail_cache_decision_add(struct mail_cache_view *view, uint32_t seq,
2999
1d328531c466 mail_cache_decision_*() prototypes were wrong.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
204 unsigned int field);
2200
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
205
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
206 int mail_cache_expunge_handler(struct mail_index_sync_map_ctx *sync_ctx,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
207 uint32_t seq, const void *data, void **context);
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
208 int mail_cache_sync_handler(struct mail_index_sync_map_ctx *sync_ctx,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
209 uint32_t seq, void *old_data, const void *new_data,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
210 void **context);
2934
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index
Timo Sirainen <tss@iki.fi>
parents: 2917
diff changeset
211 void mail_cache_sync_lost_handler(struct mail_index *index);
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
212
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
213 void mail_cache_set_syscall_error(struct mail_cache *cache,
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
214 const char *function);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
215
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
216 #endif