Mercurial > dovecot > core-2.2

annotate src/lib-index/mail-cache-private.h @ 15527:55e84bb452e6

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
lib-index: Added v2.2+ forwards compatibility to dovecot.index.cache
author Timo Sirainen <tss@iki.fi>
date Tue, 27 Nov 2012 09:53:33 +0200
parents 51f2be9aa8ad
children 13e74bd5ac8c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6410
e4eb71ae8e96 Changed .h ifdef/defines to use <NAME>_H format.
Timo Sirainen <tss@iki.fi>
parents: 6296
diff changeset
1 #ifndef MAIL_CACHE_PRIVATE_H
e4eb71ae8e96 Changed .h ifdef/defines to use <NAME>_H format.
Timo Sirainen <tss@iki.fi>
parents: 6296
diff changeset
2 #define MAIL_CACHE_PRIVATE_H
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
3
3106
9c4aa309dbac Changed dotlocking API.
Timo Sirainen <tss@iki.fi>
parents: 2999
diff changeset
4 #include "file-dotlock.h"
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
5 #include "mail-index-private.h"
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
6 #include "mail-cache.h"
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
7
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
8 #define MAIL_CACHE_VERSION 1
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
9
6296
205ee38f10d1 Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents: 6005
diff changeset
10 /* Drop fields that haven't been accessed for n seconds */
205ee38f10d1 Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents: 6005
diff changeset
11 #define MAIL_CACHE_FIELD_DROP_SECS (3600*24*30)
205ee38f10d1 Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents: 6005
diff changeset
12
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
13 /* Never compress the file if it's smaller than this */
4928
bb4ec18672fe Prefix COMPRESS_* defines with MAIL_CACHE_.
Timo Sirainen <tss@iki.fi>
parents: 4906
diff changeset
14 #define MAIL_CACHE_COMPRESS_MIN_SIZE (1024*50)
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
15
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
16 /* Don't bother remembering holes smaller than this */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
17 #define MAIL_CACHE_MIN_HOLE_SIZE 1024
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
18
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
19 /* Compress the file when deleted space reaches n% of total size */
4928
bb4ec18672fe Prefix COMPRESS_* defines with MAIL_CACHE_.
Timo Sirainen <tss@iki.fi>
parents: 4906
diff changeset
20 #define MAIL_CACHE_COMPRESS_PERCENTAGE 20
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
21
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
22 /* Compress the file when n% of rows contain continued rows.
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
23 200% means that there's 2 continued rows per record. */
4928
bb4ec18672fe Prefix COMPRESS_* defines with MAIL_CACHE_.
Timo Sirainen <tss@iki.fi>
parents: 4906
diff changeset
24 #define MAIL_CACHE_COMPRESS_CONTINUED_PERCENTAGE 200
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
25
6695
76c0e3a4df7a If we need to do more than 4 reads to get to the cached fields header,
Timo Sirainen <tss@iki.fi>
parents: 6455
diff changeset
26 /* Compress the file when we need to follow more than n next_offsets to find
76c0e3a4df7a If we need to do more than 4 reads to get to the cached fields header,
Timo Sirainen <tss@iki.fi>
parents: 6455
diff changeset
27 the latest cache header. */
76c0e3a4df7a If we need to do more than 4 reads to get to the cached fields header,
Timo Sirainen <tss@iki.fi>
parents: 6455
diff changeset
28 #define MAIL_CACHE_HEADER_FIELD_CONTINUE_COUNT 4
76c0e3a4df7a If we need to do more than 4 reads to get to the cached fields header,
Timo Sirainen <tss@iki.fi>
parents: 6455
diff changeset
29
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
30 /* Initial size for the file */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
31 #define MAIL_CACHE_INITIAL_SIZE (sizeof(struct mail_cache_header) + 10240)
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
32
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
33 /* When more space is needed, grow the file n% larger than the previous size */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
34 #define MAIL_CACHE_GROW_PERCENTAGE 10
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
35
2341
b3c0ed666757 Limit how much a single transaction can reserve space
Timo Sirainen <tss@iki.fi>
parents: 2339
diff changeset
36 /* When allocating space for transactions, don't use blocks larger than this. */
b3c0ed666757 Limit how much a single transaction can reserve space
Timo Sirainen <tss@iki.fi>
parents: 2339
diff changeset
37 #define MAIL_CACHE_MAX_RESERVED_BLOCK_SIZE (1024*512)
b3c0ed666757 Limit how much a single transaction can reserve space
Timo Sirainen <tss@iki.fi>
parents: 2339
diff changeset
38
8190
9902de77194e Mail cache: If locking failed, the next time try to lock it only once.
Timo Sirainen <tss@iki.fi>
parents: 8188
diff changeset
39 #define MAIL_CACHE_LOCK_TIMEOUT 10
8182
4bfe2c1c375f Mail cache file: Changed lock timeout to just 2 seconds.
Timo Sirainen <tss@iki.fi>
parents: 8181
diff changeset
40 #define MAIL_CACHE_LOCK_CHANGE_TIMEOUT 300
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
41
2275
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
42 #define MAIL_CACHE_IS_UNUSABLE(cache) \
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
43 ((cache)->hdr == NULL)
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
44
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
45 struct mail_cache_header {
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
46 /* version is increased only when you can't have backwards
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
47 compatibility. */
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
48 uint8_t version;
3819
61f0e1fe9b55 Moved uoff_t and time_t compatibility checks to cache file, since they're
Timo Sirainen <tss@iki.fi>
parents: 3627
diff changeset
49 uint8_t compat_sizeof_uoff_t;
15527
55e84bb452e6 lib-index: Added v2.2+ forwards compatibility to dovecot.index.cache
Timo Sirainen <tss@iki.fi>
parents: 15313
diff changeset
50 uint8_t minor_version;
55e84bb452e6 lib-index: Added v2.2+ forwards compatibility to dovecot.index.cache
Timo Sirainen <tss@iki.fi>
parents: 15313
diff changeset
51 uint8_t unused;
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
52
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
53 uint32_t indexid;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
54 uint32_t file_seq;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
55
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
56 uint32_t continued_record_count;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
57
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
58 uint32_t hole_offset;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
59 uint32_t used_file_size;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
60 uint32_t deleted_space;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
61
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
62 uint32_t field_header_offset;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
63 };
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
64
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
65 struct mail_cache_header_fields {
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
66 uint32_t next_offset;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
67 uint32_t size;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
68 uint32_t fields_count;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
69
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
70 #if 0
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
71 /* last time the field was accessed. not updated more often than
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
72 once a day. */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
73 uint32_t last_used[fields_count];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
74 /* (uint32_t)-1 for variable sized fields */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
75 uint32_t size[fields_count];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
76 /* enum mail_cache_field_type */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
77 uint8_t type[fields_count];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
78 /* enum mail_cache_decision_type */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
79 uint8_t decision[fields_count];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
80 /* NUL-separated list of field names */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
81 char name[fields_count][];
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
82 #endif
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
83 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
84
2339
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
85 #define MAIL_CACHE_FIELD_LAST_USED() \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
86 (sizeof(uint32_t) * 3)
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
87 #define MAIL_CACHE_FIELD_SIZE(count) \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
88 (MAIL_CACHE_FIELD_LAST_USED() + sizeof(uint32_t) * (count))
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
89 #define MAIL_CACHE_FIELD_TYPE(count) \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
90 (MAIL_CACHE_FIELD_SIZE(count) + sizeof(uint32_t) * (count))
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
91 #define MAIL_CACHE_FIELD_DECISION(count) \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
92 (MAIL_CACHE_FIELD_TYPE(count) + sizeof(uint8_t) * (count))
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
93 #define MAIL_CACHE_FIELD_NAMES(count) \
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
94 (MAIL_CACHE_FIELD_DECISION(count) + sizeof(uint8_t) * (count))
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
95
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
96 struct mail_cache_record {
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
97 uint32_t prev_offset;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
98 uint32_t size; /* full record size, including this header */
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
99 /* array of { uint32_t field; [ uint32_t size; ] { .. } } */
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
100 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
101
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
102 struct mail_cache_hole_header {
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
103 uint32_t next_offset; /* 0 if no holes left */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
104 uint32_t size; /* including this header */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
105
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
106 /* make sure we notice if we're treating hole as mail_cache_record.
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
107 magic is a large number so if it's treated as size field, it'll
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
108 point outside the file */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
109 #define MAIL_CACHE_HOLE_HEADER_MAGIC 0xffeedeff
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
110 uint32_t magic;
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
111 };
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
112
2339
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
113 struct mail_cache_field_private {
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
114 struct mail_cache_field field;
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
115
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
116 uint32_t uid_highwater;
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
117
6296
205ee38f10d1 Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents: 6005
diff changeset
118 /* Unused fields aren't written to cache file */
205ee38f10d1 Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents: 6005
diff changeset
119 unsigned int used:1;
7105
f0ad529ac9ea Fixes to handling when fields are dropped from cache file.
Timo Sirainen <tss@iki.fi>
parents: 7076
diff changeset
120 unsigned int adding:1;
2339
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
121 unsigned int decision_dirty:1;
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
122 };
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
123
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
124 struct mail_cache {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
125 struct mail_index *index;
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
126 uint32_t ext_id;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
127
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
128 char *filepath;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
129 int fd;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
130
6815
c991d1b132d8 Fixed cache file's NFS attribute cache flushing.
Timo Sirainen <tss@iki.fi>
parents: 6705
diff changeset
131 ino_t st_ino;
c991d1b132d8 Fixed cache file's NFS attribute cache flushing.
Timo Sirainen <tss@iki.fi>
parents: 6705
diff changeset
132 dev_t st_dev;
c991d1b132d8 Fixed cache file's NFS attribute cache flushing.
Timo Sirainen <tss@iki.fi>
parents: 6705
diff changeset
133
15313
51f2be9aa8ad lib-index: Added MAIL_INDEX_OPEN_FLAG_SAVEONLY to do only minimal reads from cache file.
Timo Sirainen <tss@iki.fi>
parents: 15312
diff changeset
134 size_t mmap_length;
51f2be9aa8ad lib-index: Added MAIL_INDEX_OPEN_FLAG_SAVEONLY to do only minimal reads from cache file.
Timo Sirainen <tss@iki.fi>
parents: 15312
diff changeset
135 /* a) mmaping the whole file */
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
136 void *mmap_base;
15313
51f2be9aa8ad lib-index: Added MAIL_INDEX_OPEN_FLAG_SAVEONLY to do only minimal reads from cache file.
Timo Sirainen <tss@iki.fi>
parents: 15312
diff changeset
137 /* b) using file cache */
2866
bf1e718e7370 Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents: 2853
diff changeset
138 struct file_cache *file_cache;
15313
51f2be9aa8ad lib-index: Added MAIL_INDEX_OPEN_FLAG_SAVEONLY to do only minimal reads from cache file.
Timo Sirainen <tss@iki.fi>
parents: 15312
diff changeset
139 /* c) using small read() calls with MAIL_INDEX_OPEN_FLAG_SAVEONLY */
51f2be9aa8ad lib-index: Added MAIL_INDEX_OPEN_FLAG_SAVEONLY to do only minimal reads from cache file.
Timo Sirainen <tss@iki.fi>
parents: 15312
diff changeset
140 uoff_t read_offset;
51f2be9aa8ad lib-index: Added MAIL_INDEX_OPEN_FLAG_SAVEONLY to do only minimal reads from cache file.
Timo Sirainen <tss@iki.fi>
parents: 15312
diff changeset
141 buffer_t *read_buf;
5736
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
142 /* mail_cache_map() increases this always. */
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
143 unsigned int remap_counter;
3106
9c4aa309dbac Changed dotlocking API.
Timo Sirainen <tss@iki.fi>
parents: 2999
diff changeset
144
9c4aa309dbac Changed dotlocking API.
Timo Sirainen <tss@iki.fi>
parents: 2999
diff changeset
145 struct dotlock_settings dotlock_settings;
9c4aa309dbac Changed dotlocking API.
Timo Sirainen <tss@iki.fi>
parents: 2999
diff changeset
146 struct dotlock *dotlock;
4876
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's
Timo Sirainen <tss@iki.fi>
parents: 4657
diff changeset
147 struct file_lock *file_lock;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
148
6703
75c48f171ad3 Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents: 6695
diff changeset
149 /* mmap_disable=no: hdr points to data / NULL when cache is invalid.
75c48f171ad3 Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents: 6695
diff changeset
150 mmap_disable=yes: hdr points to hdr_ro_copy. this is needed because
75c48f171ad3 Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents: 6695
diff changeset
151 cache invalidation can zero the data any time */
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
152 const struct mail_cache_header *hdr;
6703
75c48f171ad3 Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents: 6695
diff changeset
153 struct mail_cache_header hdr_ro_copy;
75c48f171ad3 Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents: 6695
diff changeset
154 /* hdr_copy gets updated when cache is locked and written when
75c48f171ad3 Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents: 6695
diff changeset
155 unlocking and hdr_modified=TRUE */
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
156 struct mail_cache_header hdr_copy;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
157
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
158 pool_t field_pool;
2339
406692edc12d Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents: 2327
diff changeset
159 struct mail_cache_field_private *fields;
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
160 uint32_t *field_file_map;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
161 unsigned int fields_count;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
162 struct hash_table *field_name_hash; /* name -> idx */
6705
68bb5bec043e When reading the latest fields, begin finding the fields from the last known
Timo Sirainen <tss@iki.fi>
parents: 6703
diff changeset
163 uint32_t last_field_header_offset;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
164
4247
9c3f19110538 Don't compress cache file if it was just compressed by another process.
Timo Sirainen <tss@iki.fi>
parents: 3981
diff changeset
165 /* 0 is no need for compression, otherwise the file sequence number
9c3f19110538 Don't compress cache file if it was just compressed by another process.
Timo Sirainen <tss@iki.fi>
parents: 3981
diff changeset
166 which we want compressed. */
9c3f19110538 Don't compress cache file if it was just compressed by another process.
Timo Sirainen <tss@iki.fi>
parents: 3981
diff changeset
167 uint32_t need_compress_file_seq;
9c3f19110538 Don't compress cache file if it was just compressed by another process.
Timo Sirainen <tss@iki.fi>
parents: 3981
diff changeset
168
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
169 unsigned int *file_field_map;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
170 unsigned int file_fields_count;
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
171
4655
df8de2e47fb5 Delay opening the cache file until it's actually needed.
Timo Sirainen <tss@iki.fi>
parents: 4490
diff changeset
172 unsigned int opened:1;
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
173 unsigned int locked:1;
8190
9902de77194e Mail cache: If locking failed, the next time try to lock it only once.
Timo Sirainen <tss@iki.fi>
parents: 8188
diff changeset
174 unsigned int last_lock_failed:1;
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
175 unsigned int hdr_modified:1;
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
176 unsigned int field_header_write_pending:1;
8187
f1203dce2d8c Mail cache: lock_method=dotlock could have caused a process to deadlock with itself.
Timo Sirainen <tss@iki.fi>
parents: 8182
diff changeset
177 unsigned int compressing:1;
15313
51f2be9aa8ad lib-index: Added MAIL_INDEX_OPEN_FLAG_SAVEONLY to do only minimal reads from cache file.
Timo Sirainen <tss@iki.fi>
parents: 15312
diff changeset
178 unsigned int map_with_read:1;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
179 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
180
7312
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
181 struct mail_cache_loop_track {
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
182 /* we're looping if size_sum > (max_offset-min_offset) */
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
183 uoff_t min_offset, max_offset;
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
184 uoff_t size_sum;
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
185 };
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
186
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
187 struct mail_cache_view {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
188 struct mail_cache *cache;
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
189 struct mail_index_view *view, *trans_view;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
190
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
191 struct mail_cache_transaction_ctx *transaction;
2315
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
192 uint32_t trans_seq1, trans_seq2;
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
193
7312
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
194 struct mail_cache_loop_track loop_track;
2632
ec5601f71ba0 Fix for circular record list detection, we can't use data stack for buffer.
Timo Sirainen <tss@iki.fi>
parents: 2619
diff changeset
195
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
196 /* if cached_exists_buf[field] == cached_exists_value, it's cached.
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
197 this allows us to avoid constantly clearing the whole buffer.
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
198 it needs to be cleared only when cached_exists_value is wrapped. */
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
199 buffer_t *cached_exists_buf;
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
200 uint8_t cached_exists_value;
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
201 uint32_t cached_exists_seq;
13328
4a19aea565f7 lib-index: Added mail_cache_view_update_cache_decisions()
Timo Sirainen <tss@iki.fi>
parents: 8190
diff changeset
202
4a19aea565f7 lib-index: Added mail_cache_view_update_cache_decisions()
Timo Sirainen <tss@iki.fi>
parents: 8190
diff changeset
203 unsigned int no_decision_updates:1;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
204 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
205
5736
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
206 struct mail_cache_iterate_field {
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
207 unsigned int field_idx;
15312
0d1de37ad9d8 lib-index: Code cleanup: Removed mail_cache.data
Timo Sirainen <tss@iki.fi>
parents: 15311
diff changeset
208 unsigned int size;
5736
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
209 const void *data;
15312
0d1de37ad9d8 lib-index: Code cleanup: Removed mail_cache.data
Timo Sirainen <tss@iki.fi>
parents: 15311
diff changeset
210 uoff_t offset;
5736
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
211 };
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
212
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
213 struct mail_cache_lookup_iterate_ctx {
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
214 struct mail_cache_view *view;
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
215 unsigned int remap_counter;
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
216 uint32_t seq;
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
217
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
218 const struct mail_cache_record *rec;
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
219 unsigned int pos, rec_size;
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
220 uint32_t offset;
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
221
6005
7c1e6ea4bfab mail_cache_lookup_iter_next() returned -1 if entry didn't exist, causing
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
222 unsigned int stop:1;
5736
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
223 unsigned int failed:1;
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
224 unsigned int appends_checked:1;
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
225 };
2315
22210d6cf238 Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
226
8181
5c0c674eae1a mail_cache_lock(): Return -1 if locking times out.
Timo Sirainen <tss@iki.fi>
parents: 7312
diff changeset
227 /* Explicitly lock the cache file. Returns -1 if error / timed out,
5c0c674eae1a mail_cache_lock(): Return -1 if locking times out.
Timo Sirainen <tss@iki.fi>
parents: 7312
diff changeset
228 1 if ok, 0 if cache is broken/doesn't exist */
5947
7ebe0593f488 Don't create cache file until something is actually being added to it.
Timo Sirainen <tss@iki.fi>
parents: 5736
diff changeset
229 int mail_cache_lock(struct mail_cache *cache, bool require_same_reset_id);
8188
2db48458d73b Mail cache compression: If we can't get lock immediately, do it later.
Timo Sirainen <tss@iki.fi>
parents: 8187
diff changeset
230 int mail_cache_try_lock(struct mail_cache *cache);
3627
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a
Timo Sirainen <tss@iki.fi>
parents: 3375
diff changeset
231 /* Returns -1 if cache is / just got corrupted, 0 if ok. */
c0e35566dd14 Unlocking cache file can also corrupt it since it modifies it. Added a
Timo Sirainen <tss@iki.fi>
parents: 3375
diff changeset
232 int mail_cache_unlock(struct mail_cache *cache);
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
233
3375
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
234 int mail_cache_write(struct mail_cache *cache, const void *data, size_t size,
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
235 uoff_t offset);
9e2abc4c341c Whenever writing to cache file, also updated file_cache. Fixes problems with
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
236
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
237 int mail_cache_header_fields_read(struct mail_cache *cache);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
238 int mail_cache_header_fields_update(struct mail_cache *cache);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
239 void mail_cache_header_fields_get(struct mail_cache *cache, buffer_t *dest);
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
240 int mail_cache_header_fields_get_next_offset(struct mail_cache *cache,
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2315
diff changeset
241 uint32_t *offset_r);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
242
7138
876c7bca351c Link cache records together directly when writing the new records, instead
Timo Sirainen <tss@iki.fi>
parents: 7105
diff changeset
243 uint32_t mail_cache_lookup_cur_offset(struct mail_index_view *view,
876c7bca351c Link cache records together directly when writing the new records, instead
Timo Sirainen <tss@iki.fi>
parents: 7105
diff changeset
244 uint32_t seq, uint32_t *reset_id_r);
2917
e657c7874962 Crashfixes for reading corrupted cache files.
Timo Sirainen <tss@iki.fi>
parents: 2866
diff changeset
245 int mail_cache_get_record(struct mail_cache *cache, uint32_t offset,
e657c7874962 Crashfixes for reading corrupted cache files.
Timo Sirainen <tss@iki.fi>
parents: 2866
diff changeset
246 const struct mail_cache_record **rec_r);
6841
0c970b3493ac mail_cache_field_want_add(): Return TRUE for temp fields only if we're
Timo Sirainen <tss@iki.fi>
parents: 6829
diff changeset
247 uint32_t mail_cache_get_first_new_seq(struct mail_index_view *view);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
248
7312
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
249 /* Returns TRUE if offset..size area has been tracked before.
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
250 Returns FALSE if the area may or may not have been tracked before,
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
251 but we don't know for sure yet. */
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
252 bool mail_cache_track_loops(struct mail_cache_loop_track *loop_track,
6f59434cf229 Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents: 7138
diff changeset
253 uoff_t offset, uoff_t size);
5736
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
254
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
255 /* Iterate through a message's cached fields. */
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
256 void mail_cache_lookup_iter_init(struct mail_cache_view *view, uint32_t seq,
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
257 struct mail_cache_lookup_iterate_ctx *ctx_r);
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
258 /* Returns 1 if field was returned, 0 if end of fields, or -1 if error */
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
259 int mail_cache_lookup_iter_next(struct mail_cache_lookup_iterate_ctx *ctx,
b3ed52bbeca7 Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents: 5734
diff changeset
260 struct mail_cache_iterate_field *field_r);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
261
15311
022d0d21e56d lib-index: mail_cache_map() API cleanup
Timo Sirainen <tss@iki.fi>
parents: 14083
diff changeset
262 int mail_cache_map(struct mail_cache *cache, size_t offset, size_t size,
022d0d21e56d lib-index: mail_cache_map() API cleanup
Timo Sirainen <tss@iki.fi>
parents: 14083
diff changeset
263 const void **data_r);
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
264 void mail_cache_file_close(struct mail_cache *cache);
2275
c68a3c9f6d73 Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents: 2247
diff changeset
265 int mail_cache_reopen(struct mail_cache *cache);
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
266
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
267 /* Update new_offset's prev_offset field to old_offset. */
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
268 int mail_cache_link(struct mail_cache *cache, uint32_t old_offset,
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
269 uint32_t new_offset);
2277
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
270 /* Mark record in given offset to be deleted. */
41e56f28d085 Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents: 2276
diff changeset
271 int mail_cache_delete(struct mail_cache *cache, uint32_t offset);
2276
5f374049abdb Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents: 2275
diff changeset
272
7076
3cb00348ed06 Update caching decision for fields even if it doesn't currently exist in the
Timo Sirainen <tss@iki.fi>
parents: 6841
diff changeset
273 /* Notify the decision handling code that field was looked up for seq.
3cb00348ed06 Update caching decision for fields even if it doesn't currently exist in the
Timo Sirainen <tss@iki.fi>
parents: 6841
diff changeset
274 This should be called even for fields that aren't currently in cache file */
5734
5f0832288007 Cache code cleanups
Timo Sirainen <tss@iki.fi>
parents: 5002
diff changeset
275 void mail_cache_decision_state_update(struct mail_cache_view *view,
5f0832288007 Cache code cleanups
Timo Sirainen <tss@iki.fi>
parents: 5002
diff changeset
276 uint32_t seq, unsigned int field);
2298
5beb0c20b6e8 Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents: 2277
diff changeset
277 void mail_cache_decision_add(struct mail_cache_view *view, uint32_t seq,
2999
1d328531c466 mail_cache_decision_*() prototypes were wrong.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
278 unsigned int field);
2200
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
279
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
280 int mail_cache_expunge_handler(struct mail_index_sync_map_ctx *sync_ctx,
4490
56d17f2e1d66 Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents: 4451
diff changeset
281 uint32_t seq, const void *data,
56d17f2e1d66 Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents: 4451
diff changeset
282 void **sync_context, void *context);
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
283 int mail_cache_sync_handler(struct mail_index_sync_map_ctx *sync_ctx,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
284 uint32_t seq, void *old_data, const void *new_data,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
285 void **context);
2934
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index
Timo Sirainen <tss@iki.fi>
parents: 2917
diff changeset
286 void mail_cache_sync_lost_handler(struct mail_index *index);
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2632
diff changeset
287
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
288 void mail_cache_set_syscall_error(struct mail_cache *cache,
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
289 const char *function);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
290
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
291 #endif