Mercurial > dovecot > core-2.2

annotate src/lib-index/mail-cache-private.h @ 2247:2d51bb58a070 HEAD

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
author Timo Sirainen <tss@iki.fi>
date Mon, 28 Jun 2004 20:35:27 +0300
parents 7bdef5ea4591
children c68a3c9f6d73
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1 #ifndef __MAIL_CACHE_PRIVATE_H
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
2 #define __MAIL_CACHE_PRIVATE_H
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
3
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
4 #include "mail-index-private.h"
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
5 #include "mail-cache.h"
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
6
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
7 /* Never compress the file if it's smaller than this */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
8 #define COMPRESS_MIN_SIZE (1024*50)
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
9
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
10 /* Compress the file when deleted space reaches n% of total size */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
11 #define COMPRESS_PERCENTAGE 20
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
12
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
13 /* Compress the file when n% of rows contain continued rows.
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
14 200% means that there's 2 continued rows per record. */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
15 #define COMPRESS_CONTINUED_PERCENTAGE 200
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
16
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
17 /* Initial size for the file */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
18 #define MAIL_CACHE_INITIAL_SIZE (sizeof(struct mail_cache_header) + 10240)
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
19
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
20 /* When more space is needed, grow the file n% larger than the previous size */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
21 #define MAIL_CACHE_GROW_PERCENTAGE 10
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
22
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
23 #define MAIL_CACHE_LOCK_TIMEOUT 120
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
24 #define MAIL_CACHE_LOCK_CHANGE_TIMEOUT 60
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
25 #define MAIL_CACHE_LOCK_IMMEDIATE_TIMEOUT (5*60)
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
26
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
27 #define CACHE_RECORD(cache, offset) \
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
28 ((struct mail_cache_record *) ((char *) (cache)->mmap_base + offset))
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
29
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
30 enum mail_cache_decision_type {
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
31 /* Not needed currently */
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
32 MAIL_CACHE_DECISION_NO = 0x00,
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
33 /* Needed only for new mails. Drop when compressing. */
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
34 MAIL_CACHE_DECISION_TEMP = 0x01,
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
35 /* Needed. */
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
36 MAIL_CACHE_DECISION_YES = 0x02,
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
37
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
38 /* This decision has been forced manually, don't change it. */
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
39 MAIL_CACHE_DECISION_FORCED = 0x80
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
40 };
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
41
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
42 struct mail_cache_header {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
43 uint32_t indexid;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
44 uint32_t file_seq;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
45
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
46 uint32_t continued_record_count;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
47
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
48 uint32_t used_file_size;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
49 uint32_t deleted_space;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
50
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
51 uint32_t field_usage_last_used[32]; /* time_t */
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
52 uint8_t field_usage_decision_type[32];
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
53
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
54 uint32_t header_offsets[MAIL_CACHE_HEADERS_COUNT];
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
55 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
56
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
57 struct mail_cache_record {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
58 uint32_t fields; /* enum mail_cache_field */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
59 uint32_t next_offset;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
60 uint32_t size; /* full record size, including this header */
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
61 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
62
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
63 struct mail_cache {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
64 struct mail_index *index;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
65
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
66 char *filepath;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
67 int fd;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
68
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
69 void *mmap_base;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
70 size_t mmap_length;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
71
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
72 struct mail_cache_header *hdr;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
73
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
74 pool_t split_header_pool;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
75 uint32_t split_offsets[MAIL_CACHE_HEADERS_COUNT];
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
76 const char *const *split_headers[MAIL_CACHE_HEADERS_COUNT];
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
77
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
78 enum mail_cache_field default_cache_fields;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
79 enum mail_cache_field never_cache_fields;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
80
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
81 uint32_t field_usage_uid_highwater[32];
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
82
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
83 struct mail_cache_transaction_ctx *trans_ctx;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
84 unsigned int locks;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
85
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
86 unsigned int mmap_refresh:1;
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
87 unsigned int need_compress:1;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
88 unsigned int silent:1;
2201
7bdef5ea4591 Several fixes and cleanups to cache file code, still badly broken
Timo Sirainen <tss@iki.fi>
parents: 2200
diff changeset
89 unsigned int disabled:1;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
90 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
91
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
92 struct mail_cache_view {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
93 struct mail_cache *cache;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
94 struct mail_index_view *view;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
95
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
96 unsigned int broken:1;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
97 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
98
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
99 extern unsigned int mail_cache_field_sizes[32];
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
100 extern enum mail_cache_field mail_cache_header_fields[MAIL_CACHE_HEADERS_COUNT];
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
101
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
102 uint32_t mail_cache_uint32_to_offset(uint32_t offset);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
103 uint32_t mail_cache_offset_to_uint32(uint32_t offset);
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
104 unsigned int mail_cache_field_index(enum mail_cache_field field);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
105
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
106 const char *
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
107 mail_cache_get_header_fields_str(struct mail_cache *cache, unsigned int idx);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
108 const char *const *
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
109 mail_cache_split_header(struct mail_cache *cache, const char *header);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
110
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
111 struct mail_cache_record *
2200
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
112 mail_cache_get_record(struct mail_cache *cache, uint32_t offset,
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
113 int index_offset);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
114 struct mail_cache_record *
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
115 mail_cache_get_next_record(struct mail_cache *cache,
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
116 struct mail_cache_record *rec);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
117
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
118 struct mail_cache_record *
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
119 mail_cache_lookup(struct mail_cache_view *view, uint32_t seq,
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
120 enum mail_cache_field fields);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
121
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
122 int
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
123 mail_cache_transaction_autocommit(struct mail_cache_view *view,
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
124 uint32_t seq, enum mail_cache_field fields);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
125
2200
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
126 int mail_cache_mmap_update(struct mail_cache *cache,
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
127 size_t offset, size_t size);
2247
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
128 void mail_cache_file_close(struct mail_cache *cache);
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
129
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
130 void mail_cache_handle_decisions(struct mail_cache_view *view, uint32_t seq,
2d51bb58a070 Added some smartness for deciding what to cache. Cache compression code compiles, but untested.
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
131 enum mail_cache_field field);
2200
97bb7b127617 Beginnings of getting cache file working again. Easy to break currently, but
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
132
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
133 void mail_cache_set_syscall_error(struct mail_cache *cache,
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
134 const char *function);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
135
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
136 #endif