Mercurial > dovecot > core-2.2
annotate src/lib-index/mail-cache-private.h @ 15190:60c4815778fb
lib-index: Simplified writing to dovecot.index.cache file.
The old method was basically:
- write max. 32 kB to internal buffer
- flush it by writing to reserved areas (with no locks)
The reserved areas were acquired by doing (whenever needed):
- lock dovecot.index.cache
- reserve data from dovecot.index.cache for writing, potentially increasing
the file size by writing 0 bytes. the reserved area size varies.
- unlock dovecot.index.cache
This worked, but if multiple processes were writing to the cache file it
could have left incomplete reserved areas as holes. The holes were attempted
to be filled if they were large enough.
The new method is:
- write max. 256 kB to internal buffer
- lock dovecot.index.cache
- append the buffer to dovecot.index.cache
- unlock dovecot.index.cache
No reserved areas, holes or anything else weird going on.
Ideally no data would be overwritten in the dovecot.index.cache file, only
appended. Unfortunately currently some data is still overwritten:
- mail_cache_header.{deleted_space,continued_record_count}
- mail_cache_header_fields.next_offset when writing a new one
- mail_cache_header_fields.{last_used,decision}
- mail_cache_record.prev_offset
The changing headers could eventually be moved to dovecot.index. This
however is a backwards-incompatible change. The record's prev_offset could
maybe simply just not be written in those (somewhat rare) problematic
situations.
author | Timo Sirainen <tss@iki.fi> |
---|---|
date | Thu, 04 Oct 2012 02:34:53 +0300 |
parents | 96fd2c3bf932 |
children | 1b46c1bf9d1e |
rev | line source |
---|---|
6410
e4eb71ae8e96
Changed .h ifdef/defines to use <NAME>_H format.
Timo Sirainen <tss@iki.fi>
parents:
6296
diff
changeset
|
1 #ifndef MAIL_CACHE_PRIVATE_H |
e4eb71ae8e96
Changed .h ifdef/defines to use <NAME>_H format.
Timo Sirainen <tss@iki.fi>
parents:
6296
diff
changeset
|
2 #define MAIL_CACHE_PRIVATE_H |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
3 |
3106 | 4 #include "file-dotlock.h" |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
5 #include "mail-index-private.h" |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
6 #include "mail-cache.h" |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
7 |
2276
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
8 #define MAIL_CACHE_VERSION 1 |
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
9 |
6296
205ee38f10d1
Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents:
6005
diff
changeset
|
10 /* Drop fields that haven't been accessed for n seconds */ |
205ee38f10d1
Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents:
6005
diff
changeset
|
11 #define MAIL_CACHE_FIELD_DROP_SECS (3600*24*30) |
205ee38f10d1
Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents:
6005
diff
changeset
|
12 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
13 /* Never compress the file if it's smaller than this */ |
4928
bb4ec18672fe
Prefix COMPRESS_* defines with MAIL_CACHE_.
Timo Sirainen <tss@iki.fi>
parents:
4906
diff
changeset
|
14 #define MAIL_CACHE_COMPRESS_MIN_SIZE (1024*50) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
15 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
16 /* Compress the file when deleted space reaches n% of total size */ |
4928
bb4ec18672fe
Prefix COMPRESS_* defines with MAIL_CACHE_.
Timo Sirainen <tss@iki.fi>
parents:
4906
diff
changeset
|
17 #define MAIL_CACHE_COMPRESS_PERCENTAGE 20 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
18 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
19 /* Compress the file when n% of rows contain continued rows. |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
20 200% means that there's 2 continued rows per record. */ |
4928
bb4ec18672fe
Prefix COMPRESS_* defines with MAIL_CACHE_.
Timo Sirainen <tss@iki.fi>
parents:
4906
diff
changeset
|
21 #define MAIL_CACHE_COMPRESS_CONTINUED_PERCENTAGE 200 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
22 |
6695
76c0e3a4df7a
If we need to do more than 4 reads to get to the cached fields header,
Timo Sirainen <tss@iki.fi>
parents:
6455
diff
changeset
|
23 /* Compress the file when we need to follow more than n next_offsets to find |
76c0e3a4df7a
If we need to do more than 4 reads to get to the cached fields header,
Timo Sirainen <tss@iki.fi>
parents:
6455
diff
changeset
|
24 the latest cache header. */ |
76c0e3a4df7a
If we need to do more than 4 reads to get to the cached fields header,
Timo Sirainen <tss@iki.fi>
parents:
6455
diff
changeset
|
25 #define MAIL_CACHE_HEADER_FIELD_CONTINUE_COUNT 4 |
76c0e3a4df7a
If we need to do more than 4 reads to get to the cached fields header,
Timo Sirainen <tss@iki.fi>
parents:
6455
diff
changeset
|
26 |
8190
9902de77194e
Mail cache: If locking failed, the next time try to lock it only once.
Timo Sirainen <tss@iki.fi>
parents:
8188
diff
changeset
|
27 #define MAIL_CACHE_LOCK_TIMEOUT 10 |
8182
4bfe2c1c375f
Mail cache file: Changed lock timeout to just 2 seconds.
Timo Sirainen <tss@iki.fi>
parents:
8181
diff
changeset
|
28 #define MAIL_CACHE_LOCK_CHANGE_TIMEOUT 300 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
29 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
30 #define CACHE_RECORD(cache, offset) \ |
2866
bf1e718e7370
Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents:
2853
diff
changeset
|
31 ((const struct mail_cache_record *) \ |
bf1e718e7370
Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents:
2853
diff
changeset
|
32 ((const char *) (cache)->data + offset)) |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
33 |
2275
c68a3c9f6d73
Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents:
2247
diff
changeset
|
34 #define MAIL_CACHE_IS_UNUSABLE(cache) \ |
c68a3c9f6d73
Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents:
2247
diff
changeset
|
35 ((cache)->hdr == NULL) |
c68a3c9f6d73
Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents:
2247
diff
changeset
|
36 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
37 struct mail_cache_header { |
2276
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
38 /* version is increased only when you can't have backwards |
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
39 compatibility. */ |
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
40 uint8_t version; |
3819
61f0e1fe9b55
Moved uoff_t and time_t compatibility checks to cache file, since they're
Timo Sirainen <tss@iki.fi>
parents:
3627
diff
changeset
|
41 uint8_t compat_sizeof_uoff_t; |
4657
a2a9b6a92831
Don't use time_t anymore in cache file. uint32_t is large enough and makes
Timo Sirainen <tss@iki.fi>
parents:
4655
diff
changeset
|
42 uint8_t unused[2]; |
2276
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
43 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
44 uint32_t indexid; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
45 uint32_t file_seq; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
46 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
47 uint32_t continued_record_count; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
48 |
15190
60c4815778fb
lib-index: Simplified writing to dovecot.index.cache file.
Timo Sirainen <tss@iki.fi>
parents:
14923
diff
changeset
|
49 uint32_t unused_old_hole_offset; |
60c4815778fb
lib-index: Simplified writing to dovecot.index.cache file.
Timo Sirainen <tss@iki.fi>
parents:
14923
diff
changeset
|
50 uint32_t backwards_compat_used_file_size; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
51 uint32_t deleted_space; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
52 |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
53 uint32_t field_header_offset; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
54 }; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
55 |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
56 struct mail_cache_header_fields { |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
57 uint32_t next_offset; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
58 uint32_t size; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
59 uint32_t fields_count; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
60 |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
61 #if 0 |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
62 /* last time the field was accessed. not updated more often than |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
63 once a day. */ |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
64 uint32_t last_used[fields_count]; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
65 /* (uint32_t)-1 for variable sized fields */ |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
66 uint32_t size[fields_count]; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
67 /* enum mail_cache_field_type */ |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
68 uint8_t type[fields_count]; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
69 /* enum mail_cache_decision_type */ |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
70 uint8_t decision[fields_count]; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
71 /* NUL-separated list of field names */ |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
72 char name[fields_count][]; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
73 #endif |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
74 }; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
75 |
2339
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
76 #define MAIL_CACHE_FIELD_LAST_USED() \ |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
77 (sizeof(uint32_t) * 3) |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
78 #define MAIL_CACHE_FIELD_SIZE(count) \ |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
79 (MAIL_CACHE_FIELD_LAST_USED() + sizeof(uint32_t) * (count)) |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
80 #define MAIL_CACHE_FIELD_TYPE(count) \ |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
81 (MAIL_CACHE_FIELD_SIZE(count) + sizeof(uint32_t) * (count)) |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
82 #define MAIL_CACHE_FIELD_DECISION(count) \ |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
83 (MAIL_CACHE_FIELD_TYPE(count) + sizeof(uint8_t) * (count)) |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
84 #define MAIL_CACHE_FIELD_NAMES(count) \ |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
85 (MAIL_CACHE_FIELD_DECISION(count) + sizeof(uint8_t) * (count)) |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
86 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
87 struct mail_cache_record { |
2276
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
88 uint32_t prev_offset; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
89 uint32_t size; /* full record size, including this header */ |
2298
5beb0c20b6e8
Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents:
2277
diff
changeset
|
90 /* array of { uint32_t field; [ uint32_t size; ] { .. } } */ |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
91 }; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
92 |
2339
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
93 struct mail_cache_field_private { |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
94 struct mail_cache_field field; |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
95 |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
96 uint32_t uid_highwater; |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
97 |
6296
205ee38f10d1
Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents:
6005
diff
changeset
|
98 /* Unused fields aren't written to cache file */ |
205ee38f10d1
Drop fields that haven't been used for 30 days when compressing.
Timo Sirainen <tss@iki.fi>
parents:
6005
diff
changeset
|
99 unsigned int used:1; |
7105
f0ad529ac9ea
Fixes to handling when fields are dropped from cache file.
Timo Sirainen <tss@iki.fi>
parents:
7076
diff
changeset
|
100 unsigned int adding:1; |
2339
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
101 unsigned int decision_dirty:1; |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
102 }; |
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
103 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
104 struct mail_cache { |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
105 struct mail_index *index; |
2853
512dd7d76cdc
Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents:
2632
diff
changeset
|
106 uint32_t ext_id; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
107 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
108 char *filepath; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
109 int fd; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
110 |
6815
c991d1b132d8
Fixed cache file's NFS attribute cache flushing.
Timo Sirainen <tss@iki.fi>
parents:
6705
diff
changeset
|
111 ino_t st_ino; |
c991d1b132d8
Fixed cache file's NFS attribute cache flushing.
Timo Sirainen <tss@iki.fi>
parents:
6705
diff
changeset
|
112 dev_t st_dev; |
c991d1b132d8
Fixed cache file's NFS attribute cache flushing.
Timo Sirainen <tss@iki.fi>
parents:
6705
diff
changeset
|
113 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
114 void *mmap_base; |
2866
bf1e718e7370
Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents:
2853
diff
changeset
|
115 const void *data; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
116 size_t mmap_length; |
2866
bf1e718e7370
Cache file works now with mmap_disable=yes. Still needs a few optimizations.
Timo Sirainen <tss@iki.fi>
parents:
2853
diff
changeset
|
117 struct file_cache *file_cache; |
5736
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
118 /* mail_cache_map() increases this always. */ |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
119 unsigned int remap_counter; |
3106 | 120 |
121 struct dotlock_settings dotlock_settings; | |
122 struct dotlock *dotlock; | |
4876
f1d77064884c
Lock handling changes. Everything goes through file-lock API now and there's
Timo Sirainen <tss@iki.fi>
parents:
4657
diff
changeset
|
123 struct file_lock *file_lock; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
124 |
6703
75c48f171ad3
Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents:
6695
diff
changeset
|
125 /* mmap_disable=no: hdr points to data / NULL when cache is invalid. |
75c48f171ad3
Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents:
6695
diff
changeset
|
126 mmap_disable=yes: hdr points to hdr_ro_copy. this is needed because |
75c48f171ad3
Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents:
6695
diff
changeset
|
127 cache invalidation can zero the data any time */ |
2277
41e56f28d085
Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents:
2276
diff
changeset
|
128 const struct mail_cache_header *hdr; |
6703
75c48f171ad3
Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents:
6695
diff
changeset
|
129 struct mail_cache_header hdr_ro_copy; |
75c48f171ad3
Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents:
6695
diff
changeset
|
130 /* hdr_copy gets updated when cache is locked and written when |
75c48f171ad3
Keep a separate copy of mail_cache_header when mmap_disable=yes, otherwise
Timo Sirainen <tss@iki.fi>
parents:
6695
diff
changeset
|
131 unlocking and hdr_modified=TRUE */ |
2277
41e56f28d085
Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents:
2276
diff
changeset
|
132 struct mail_cache_header hdr_copy; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
133 |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
134 pool_t field_pool; |
2339
406692edc12d
Cache fixes. Decisions are saved again.
Timo Sirainen <tss@iki.fi>
parents:
2327
diff
changeset
|
135 struct mail_cache_field_private *fields; |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
136 uint32_t *field_file_map; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
137 unsigned int fields_count; |
14923
96fd2c3bf932
Reverted "support for non-pointers" part of the hash table API changes.
Timo Sirainen <tss@iki.fi>
parents:
14918
diff
changeset
|
138 HASH_TABLE(char *, void *) field_name_hash; /* name -> idx */ |
6705
68bb5bec043e
When reading the latest fields, begin finding the fields from the last known
Timo Sirainen <tss@iki.fi>
parents:
6703
diff
changeset
|
139 uint32_t last_field_header_offset; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
140 |
4247
9c3f19110538
Don't compress cache file if it was just compressed by another process.
Timo Sirainen <tss@iki.fi>
parents:
3981
diff
changeset
|
141 /* 0 is no need for compression, otherwise the file sequence number |
9c3f19110538
Don't compress cache file if it was just compressed by another process.
Timo Sirainen <tss@iki.fi>
parents:
3981
diff
changeset
|
142 which we want compressed. */ |
9c3f19110538
Don't compress cache file if it was just compressed by another process.
Timo Sirainen <tss@iki.fi>
parents:
3981
diff
changeset
|
143 uint32_t need_compress_file_seq; |
9c3f19110538
Don't compress cache file if it was just compressed by another process.
Timo Sirainen <tss@iki.fi>
parents:
3981
diff
changeset
|
144 |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
145 unsigned int *file_field_map; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
146 unsigned int file_fields_count; |
2247
2d51bb58a070
Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents:
2201
diff
changeset
|
147 |
4655
df8de2e47fb5
Delay opening the cache file until it's actually needed.
Timo Sirainen <tss@iki.fi>
parents:
4490
diff
changeset
|
148 unsigned int opened:1; |
2277
41e56f28d085
Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents:
2276
diff
changeset
|
149 unsigned int locked:1; |
8190
9902de77194e
Mail cache: If locking failed, the next time try to lock it only once.
Timo Sirainen <tss@iki.fi>
parents:
8188
diff
changeset
|
150 unsigned int last_lock_failed:1; |
2277
41e56f28d085
Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents:
2276
diff
changeset
|
151 unsigned int hdr_modified:1; |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
152 unsigned int field_header_write_pending:1; |
8187
f1203dce2d8c
Mail cache: lock_method=dotlock could have caused a process to deadlock with itself.
Timo Sirainen <tss@iki.fi>
parents:
8182
diff
changeset
|
153 unsigned int compressing:1; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
154 }; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
155 |
7312
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
156 struct mail_cache_loop_track { |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
157 /* we're looping if size_sum > (max_offset-min_offset) */ |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
158 uoff_t min_offset, max_offset; |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
159 uoff_t size_sum; |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
160 }; |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
161 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
162 struct mail_cache_view { |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
163 struct mail_cache *cache; |
2853
512dd7d76cdc
Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents:
2632
diff
changeset
|
164 struct mail_index_view *view, *trans_view; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
165 |
2298
5beb0c20b6e8
Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents:
2277
diff
changeset
|
166 struct mail_cache_transaction_ctx *transaction; |
2315
22210d6cf238
Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents:
2298
diff
changeset
|
167 uint32_t trans_seq1, trans_seq2; |
22210d6cf238
Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents:
2298
diff
changeset
|
168 |
7312
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
169 struct mail_cache_loop_track loop_track; |
2632
ec5601f71ba0
Fix for circular record list detection, we can't use data stack for buffer.
Timo Sirainen <tss@iki.fi>
parents:
2619
diff
changeset
|
170 |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
171 /* if cached_exists_buf[field] == cached_exists_value, it's cached. |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
172 this allows us to avoid constantly clearing the whole buffer. |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
173 it needs to be cleared only when cached_exists_value is wrapped. */ |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
174 buffer_t *cached_exists_buf; |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
175 uint8_t cached_exists_value; |
2298
5beb0c20b6e8
Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents:
2277
diff
changeset
|
176 uint32_t cached_exists_seq; |
13328
4a19aea565f7
lib-index: Added mail_cache_view_update_cache_decisions()
Timo Sirainen <tss@iki.fi>
parents:
8190
diff
changeset
|
177 |
4a19aea565f7
lib-index: Added mail_cache_view_update_cache_decisions()
Timo Sirainen <tss@iki.fi>
parents:
8190
diff
changeset
|
178 unsigned int no_decision_updates:1; |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
179 }; |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
180 |
5736
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
181 struct mail_cache_iterate_field { |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
182 unsigned int field_idx; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
183 const void *data; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
184 unsigned int size; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
185 }; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
186 |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
187 struct mail_cache_lookup_iterate_ctx { |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
188 struct mail_cache_view *view; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
189 unsigned int remap_counter; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
190 uint32_t seq; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
191 |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
192 const struct mail_cache_record *rec; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
193 unsigned int pos, rec_size; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
194 uint32_t offset; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
195 |
6005
7c1e6ea4bfab
mail_cache_lookup_iter_next() returned -1 if entry didn't exist, causing
Timo Sirainen <tss@iki.fi>
parents:
5972
diff
changeset
|
196 unsigned int stop:1; |
5736
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
197 unsigned int failed:1; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
198 unsigned int appends_checked:1; |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
199 }; |
2315
22210d6cf238
Cache fixes. Lookups now look into transactions too.
Timo Sirainen <tss@iki.fi>
parents:
2298
diff
changeset
|
200 |
8181
5c0c674eae1a
mail_cache_lock(): Return -1 if locking times out.
Timo Sirainen <tss@iki.fi>
parents:
7312
diff
changeset
|
201 /* Explicitly lock the cache file. Returns -1 if error / timed out, |
5c0c674eae1a
mail_cache_lock(): Return -1 if locking times out.
Timo Sirainen <tss@iki.fi>
parents:
7312
diff
changeset
|
202 1 if ok, 0 if cache is broken/doesn't exist */ |
5947
7ebe0593f488
Don't create cache file until something is actually being added to it.
Timo Sirainen <tss@iki.fi>
parents:
5736
diff
changeset
|
203 int mail_cache_lock(struct mail_cache *cache, bool require_same_reset_id); |
8188
2db48458d73b
Mail cache compression: If we can't get lock immediately, do it later.
Timo Sirainen <tss@iki.fi>
parents:
8187
diff
changeset
|
204 int mail_cache_try_lock(struct mail_cache *cache); |
3627
c0e35566dd14
Unlocking cache file can also corrupt it since it modifies it. Added a
Timo Sirainen <tss@iki.fi>
parents:
3375
diff
changeset
|
205 /* Returns -1 if cache is / just got corrupted, 0 if ok. */ |
c0e35566dd14
Unlocking cache file can also corrupt it since it modifies it. Added a
Timo Sirainen <tss@iki.fi>
parents:
3375
diff
changeset
|
206 int mail_cache_unlock(struct mail_cache *cache); |
2277
41e56f28d085
Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents:
2276
diff
changeset
|
207 |
3375
9e2abc4c341c
Whenever writing to cache file, also updated file_cache. Fixes problems with
Timo Sirainen <tss@iki.fi>
parents:
3191
diff
changeset
|
208 int mail_cache_write(struct mail_cache *cache, const void *data, size_t size, |
9e2abc4c341c
Whenever writing to cache file, also updated file_cache. Fixes problems with
Timo Sirainen <tss@iki.fi>
parents:
3191
diff
changeset
|
209 uoff_t offset); |
15190
60c4815778fb
lib-index: Simplified writing to dovecot.index.cache file.
Timo Sirainen <tss@iki.fi>
parents:
14923
diff
changeset
|
210 int mail_cache_append(struct mail_cache *cache, const void *data, size_t size, |
60c4815778fb
lib-index: Simplified writing to dovecot.index.cache file.
Timo Sirainen <tss@iki.fi>
parents:
14923
diff
changeset
|
211 uint32_t *offset_r); |
3375
9e2abc4c341c
Whenever writing to cache file, also updated file_cache. Fixes problems with
Timo Sirainen <tss@iki.fi>
parents:
3191
diff
changeset
|
212 |
2327
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
213 int mail_cache_header_fields_read(struct mail_cache *cache); |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
214 int mail_cache_header_fields_update(struct mail_cache *cache); |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
215 void mail_cache_header_fields_get(struct mail_cache *cache, buffer_t *dest); |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
216 int mail_cache_header_fields_get_next_offset(struct mail_cache *cache, |
7d02e2a7672d
Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents:
2315
diff
changeset
|
217 uint32_t *offset_r); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
218 |
7138
876c7bca351c
Link cache records together directly when writing the new records, instead
Timo Sirainen <tss@iki.fi>
parents:
7105
diff
changeset
|
219 uint32_t mail_cache_lookup_cur_offset(struct mail_index_view *view, |
876c7bca351c
Link cache records together directly when writing the new records, instead
Timo Sirainen <tss@iki.fi>
parents:
7105
diff
changeset
|
220 uint32_t seq, uint32_t *reset_id_r); |
2917
e657c7874962
Crashfixes for reading corrupted cache files.
Timo Sirainen <tss@iki.fi>
parents:
2866
diff
changeset
|
221 int mail_cache_get_record(struct mail_cache *cache, uint32_t offset, |
e657c7874962
Crashfixes for reading corrupted cache files.
Timo Sirainen <tss@iki.fi>
parents:
2866
diff
changeset
|
222 const struct mail_cache_record **rec_r); |
6841
0c970b3493ac
mail_cache_field_want_add(): Return TRUE for temp fields only if we're
Timo Sirainen <tss@iki.fi>
parents:
6829
diff
changeset
|
223 uint32_t mail_cache_get_first_new_seq(struct mail_index_view *view); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
224 |
7312
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
225 /* Returns TRUE if offset..size area has been tracked before. |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
226 Returns FALSE if the area may or may not have been tracked before, |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
227 but we don't know for sure yet. */ |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
228 bool mail_cache_track_loops(struct mail_cache_loop_track *loop_track, |
6f59434cf229
Optimized cache record loop tracking.
Timo Sirainen <tss@iki.fi>
parents:
7138
diff
changeset
|
229 uoff_t offset, uoff_t size); |
5736
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
230 |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
231 /* Iterate through a message's cached fields. */ |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
232 void mail_cache_lookup_iter_init(struct mail_cache_view *view, uint32_t seq, |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
233 struct mail_cache_lookup_iterate_ctx *ctx_r); |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
234 /* Returns 1 if field was returned, 0 if end of fields, or -1 if error */ |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
235 int mail_cache_lookup_iter_next(struct mail_cache_lookup_iterate_ctx *ctx, |
b3ed52bbeca7
Changed mail_cache_foreach() to be an iterator: mail_cache_lookup_iter_*()
Timo Sirainen <tss@iki.fi>
parents:
5734
diff
changeset
|
236 struct mail_cache_iterate_field *field_r); |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
237 |
2275
c68a3c9f6d73
Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents:
2247
diff
changeset
|
238 int mail_cache_map(struct mail_cache *cache, size_t offset, size_t size); |
2247
2d51bb58a070
Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents:
2201
diff
changeset
|
239 void mail_cache_file_close(struct mail_cache *cache); |
2275
c68a3c9f6d73
Cache file compression works now and compressed cache file is reopened.
Timo Sirainen <tss@iki.fi>
parents:
2247
diff
changeset
|
240 int mail_cache_reopen(struct mail_cache *cache); |
2247
2d51bb58a070
Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents:
2201
diff
changeset
|
241 |
2277
41e56f28d085
Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents:
2276
diff
changeset
|
242 /* Update new_offset's prev_offset field to old_offset. */ |
2276
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
243 int mail_cache_link(struct mail_cache *cache, uint32_t old_offset, |
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
244 uint32_t new_offset); |
2277
41e56f28d085
Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents:
2276
diff
changeset
|
245 /* Mark record in given offset to be deleted. */ |
41e56f28d085
Cache updating is done now by first reserving space where to write, and then
Timo Sirainen <tss@iki.fi>
parents:
2276
diff
changeset
|
246 int mail_cache_delete(struct mail_cache *cache, uint32_t offset); |
2276
5f374049abdb
Caching fixes and optimizations. Removed all network byte ordering code -
Timo Sirainen <tss@iki.fi>
parents:
2275
diff
changeset
|
247 |
7076
3cb00348ed06
Update caching decision for fields even if it doesn't currently exist in the
Timo Sirainen <tss@iki.fi>
parents:
6841
diff
changeset
|
248 /* Notify the decision handling code that field was looked up for seq. |
3cb00348ed06
Update caching decision for fields even if it doesn't currently exist in the
Timo Sirainen <tss@iki.fi>
parents:
6841
diff
changeset
|
249 This should be called even for fields that aren't currently in cache file */ |
5734 | 250 void mail_cache_decision_state_update(struct mail_cache_view *view, |
251 uint32_t seq, unsigned int field); | |
2298
5beb0c20b6e8
Cache file fixes, API changes, etc. It's still in somewhat ugly state, but
Timo Sirainen <tss@iki.fi>
parents:
2277
diff
changeset
|
252 void mail_cache_decision_add(struct mail_cache_view *view, uint32_t seq, |
2999
1d328531c466
mail_cache_decision_*() prototypes were wrong.
Timo Sirainen <tss@iki.fi>
parents:
2941
diff
changeset
|
253 unsigned int field); |
2200
97bb7b127617
Beginnings of getting cache file working again. Easy to break currently, but
Timo Sirainen <tss@iki.fi>
parents:
1915
diff
changeset
|
254 |
2853
512dd7d76cdc
Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents:
2632
diff
changeset
|
255 int mail_cache_expunge_handler(struct mail_index_sync_map_ctx *sync_ctx, |
4490
56d17f2e1d66
Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents:
4451
diff
changeset
|
256 uint32_t seq, const void *data, |
56d17f2e1d66
Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents:
4451
diff
changeset
|
257 void **sync_context, void *context); |
2853
512dd7d76cdc
Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents:
2632
diff
changeset
|
258 int mail_cache_sync_handler(struct mail_index_sync_map_ctx *sync_ctx, |
512dd7d76cdc
Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents:
2632
diff
changeset
|
259 uint32_t seq, void *old_data, const void *new_data, |
512dd7d76cdc
Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents:
2632
diff
changeset
|
260 void **context); |
2934
c3ae75597952
Added "sync lost" handlers which are called with mmap_disable=yes when index
Timo Sirainen <tss@iki.fi>
parents:
2917
diff
changeset
|
261 void mail_cache_sync_lost_handler(struct mail_index *index); |
2853
512dd7d76cdc
Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents:
2632
diff
changeset
|
262 |
1915
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
263 void mail_cache_set_syscall_error(struct mail_cache *cache, |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
264 const char *function); |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
265 |
79790750c349
importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff
changeset
|
266 #endif |