Mercurial > dovecot > core-2.2

annotate src/lib-index/mail-index-private.h @ 22576:707ae9de3812

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
lib: istream-multiplex - Minor code cleanup Avoid propagating the error twice, and avoid any confusion about what "got" actually contains.
author Timo Sirainen <timo.sirainen@dovecot.fi>
date Thu, 05 Oct 2017 20:24:11 +0300
parents ba1a35c5ead7
children b7e049f3aa16
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6410
e4eb71ae8e96 Changed .h ifdef/defines to use <NAME>_H format.
Timo Sirainen <tss@iki.fi>
parents: 6390
diff changeset
1 #ifndef MAIL_INDEX_PRIVATE_H
e4eb71ae8e96 Changed .h ifdef/defines to use <NAME>_H format.
Timo Sirainen <tss@iki.fi>
parents: 6390
diff changeset
2 #define MAIL_INDEX_PRIVATE_H
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
3
4876
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's
Timo Sirainen <tss@iki.fi>
parents: 4662
diff changeset
4 #include "file-lock.h"
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
5 #include "mail-index.h"
9612
2cab7f501986 Moved mail_index_seq_array_*() to mail-index-util.c and added its own .h file.
Timo Sirainen <tss@iki.fi>
parents: 9537
diff changeset
6 #include "mail-index-util.h"
4453
85fcdb478608 Beginnings of joining mail-storage API more closely to mail-index, so that
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4451
diff changeset
7 #include "mail-index-view-private.h"
85fcdb478608 Beginnings of joining mail-storage API more closely to mail-index, so that
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4451
diff changeset
8 #include "mail-index-transaction-private.h"
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
9
5784
9493c7f1ebca Updating index file by overwriting changed parts didn't work correctly.
Timo Sirainen <tss@iki.fi>
parents: 5775
diff changeset
10 #include <sys/stat.h>
9493c7f1ebca Updating index file by overwriting changed parts didn't work correctly.
Timo Sirainen <tss@iki.fi>
parents: 5775
diff changeset
11
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
12 struct mail_transaction_header;
4237
20e6d554d3fe mmap_disable: When syncing in-memory index from transaction log, we didn't
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4071
diff changeset
13 struct mail_transaction_log_view;
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
14 struct mail_index_sync_map_ctx;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
15
5689
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
16 /* How large index files to mmap() instead of reading to memory. */
5867
1b70ae186611 mmaping works again properly. Changed to use private mmaps which are
Timo Sirainen <tss@iki.fi>
parents: 5854
diff changeset
17 #define MAIL_INDEX_MMAP_MIN_SIZE (1024*64)
1917
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
18 /* How many times to retry opening index files if read/fstat returns ESTALE.
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
19 This happens with NFS when the file has been deleted (ie. index file was
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for
Timo Sirainen <tss@iki.fi>
parents: 1915
diff changeset
20 rewritten by another computer than us). */
4071
cd3d26cf124a Renamed safe-open.* to nfs-workarounds.*, safe_open() to nfs_safe_open() and
Timo Sirainen <tss@iki.fi>
parents: 4023
diff changeset
21 #define MAIL_INDEX_ESTALE_RETRY_COUNT NFS_ESTALE_RETRY_COUNT
7367
8b0fe9c43c85 Don't allow too large extension header sizes. They're probably caused by
Timo Sirainen <tss@iki.fi>
parents: 6872
diff changeset
22 /* Large extension header sizes are probably caused by file corruption, so
8b0fe9c43c85 Don't allow too large extension header sizes. They're probably caused by
Timo Sirainen <tss@iki.fi>
parents: 6872
diff changeset
23 try to catch them by limiting the header size. */
8b0fe9c43c85 Don't allow too large extension header sizes. They're probably caused by
Timo Sirainen <tss@iki.fi>
parents: 6872
diff changeset
24 #define MAIL_INDEX_EXT_HEADER_MAX_SIZE (1024*1024*16-1)
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
25
8993
48d2377939c2 Write to main index file less often.
Timo Sirainen <tss@iki.fi>
parents: 8973
diff changeset
26 /* Write to main index file when bytes-to-be-read-from-log is between these
48d2377939c2 Write to main index file less often.
Timo Sirainen <tss@iki.fi>
parents: 8973
diff changeset
27 values. */
48d2377939c2 Write to main index file less often.
Timo Sirainen <tss@iki.fi>
parents: 8973
diff changeset
28 #define MAIL_INDEX_MIN_WRITE_BYTES (1024*8)
48d2377939c2 Write to main index file less often.
Timo Sirainen <tss@iki.fi>
parents: 8973
diff changeset
29 #define MAIL_INDEX_MAX_WRITE_BYTES (1024*128)
48d2377939c2 Write to main index file less often.
Timo Sirainen <tss@iki.fi>
parents: 8973
diff changeset
30
3243
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc().
Timo Sirainen <tss@iki.fi>
parents: 3231
diff changeset
31 #define MAIL_INDEX_IS_IN_MEMORY(index) \
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc().
Timo Sirainen <tss@iki.fi>
parents: 3231
diff changeset
32 ((index)->dir == NULL)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc().
Timo Sirainen <tss@iki.fi>
parents: 3231
diff changeset
33
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
34 #define MAIL_INDEX_MAP_IS_IN_MEMORY(map) \
6165
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
35 ((map)->rec_map->mmap_base == NULL)
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
36
2240
51719889ffcf Record size is allowed to change between index files. This will allow adding
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
37 #define MAIL_INDEX_MAP_IDX(map, idx) \
2114
4aedcb4c51cb Added support for per-index sized mail_index_record.
Timo Sirainen <tss@iki.fi>
parents: 2073
diff changeset
38 ((struct mail_index_record *) \
6165
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
39 PTR_OFFSET((map)->rec_map->records, (idx) * (map)->hdr.record_size))
17427
c83ef1042679 mail-index: new helper macro with more user-friendly semantics
Phil Carmody <phil@dovecot.fi>
parents: 17108
diff changeset
40 #define MAIL_INDEX_REC_AT_SEQ(map, seq) \
c83ef1042679 mail-index: new helper macro with more user-friendly semantics
Phil Carmody <phil@dovecot.fi>
parents: 17108
diff changeset
41 ((struct mail_index_record *) \
c83ef1042679 mail-index: new helper macro with more user-friendly semantics
Phil Carmody <phil@dovecot.fi>
parents: 17108
diff changeset
42 PTR_OFFSET((map)->rec_map->records, ((seq)-1) * (map)->hdr.record_size))
2240
51719889ffcf Record size is allowed to change between index files. This will allow adding
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
43
7812
d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset.
Timo Sirainen <tss@iki.fi>
parents: 7798
diff changeset
44 #define MAIL_TRANSACTION_FLAG_UPDATE_IS_INTERNAL(u) \
14599
dbd42f7198eb shared mailboxes: Per-user flags can now be stored in private index files.
Timo Sirainen <tss@iki.fi>
parents: 14231
diff changeset
45 ((((u)->add_flags | (u)->remove_flags) & MAIL_INDEX_FLAGS_MASK) == 0 && \
dbd42f7198eb shared mailboxes: Per-user flags can now be stored in private index files.
Timo Sirainen <tss@iki.fi>
parents: 14231
diff changeset
46 (u)->modseq_inc_flag == 0)
7812
d10cb44ab446 Modseqs are no longer calculated from transaction log sequence + offset.
Timo Sirainen <tss@iki.fi>
parents: 7798
diff changeset
47
8547
545cb188f7ab Replaced literal "keywords" strings with a macro.
Timo Sirainen <tss@iki.fi>
parents: 8541
diff changeset
48 #define MAIL_INDEX_EXT_KEYWORDS "keywords"
545cb188f7ab Replaced literal "keywords" strings with a macro.
Timo Sirainen <tss@iki.fi>
parents: 8541
diff changeset
49
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
50 typedef int mail_index_expunge_handler_t(struct mail_index_sync_map_ctx *ctx,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
51 uint32_t seq, const void *data,
4490
56d17f2e1d66 Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents: 4453
diff changeset
52 void **sync_context, void *context);
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
53 typedef int mail_index_sync_handler_t(struct mail_index_sync_map_ctx *ctx,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
54 uint32_t seq, void *old_data,
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
55 const void *new_data, void **context);
2934
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index
Timo Sirainen <tss@iki.fi>
parents: 2921
diff changeset
56 typedef void mail_index_sync_lost_handler_t(struct mail_index *index);
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
57
2683
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
58 #define MAIL_INDEX_HEADER_SIZE_ALIGN(size) \
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
59 (((size) + 7) & ~7)
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
60
2671
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in
Timo Sirainen <tss@iki.fi>
parents: 2563
diff changeset
61 struct mail_index_ext {
2240
51719889ffcf Record size is allowed to change between index files. This will allow adding
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
62 const char *name;
2921
a5eda271256f Wrong extension sync_handler might have been called if index extensions
Timo Sirainen <tss@iki.fi>
parents: 2911
diff changeset
63 uint32_t index_idx; /* index ext_id */
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
64 uint32_t reset_id;
6222
e6b1432e353f mail_index_map_register_ext() now takes ext_offset instead of hdr_offset.
Timo Sirainen <tss@iki.fi>
parents: 6168
diff changeset
65 uint32_t ext_offset; /* points to beginning of mail_index_ext_header */
3816
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
66 uint32_t hdr_offset; /* points to mail_index_ext_header.data[] */
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
67 uint32_t hdr_size; /* size of mail_index_ext_header.data[] */
2683
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
68 uint16_t record_offset;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
69 uint16_t record_size;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
70 uint16_t record_align;
2563
5752e5ac88f0 Save extra record/header infos into index file permanently.
Timo Sirainen <tss@iki.fi>
parents: 2407
diff changeset
71 };
5752e5ac88f0 Save extra record/header infos into index file permanently.
Timo Sirainen <tss@iki.fi>
parents: 2407
diff changeset
72
2671
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in
Timo Sirainen <tss@iki.fi>
parents: 2563
diff changeset
73 struct mail_index_ext_header {
3603
36e1899b109b Added some comments
Timo Sirainen <tss@iki.fi>
parents: 3296
diff changeset
74 uint32_t hdr_size; /* size of data[] */
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
75 uint32_t reset_id;
2683
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
76 uint16_t record_offset;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
77 uint16_t record_size;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
78 uint16_t record_align;
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
79 uint16_t name_size;
3603
36e1899b109b Added some comments
Timo Sirainen <tss@iki.fi>
parents: 3296
diff changeset
80 /* unsigned char name[name_size] */
36e1899b109b Added some comments
Timo Sirainen <tss@iki.fi>
parents: 3296
diff changeset
81 /* unsigned char data[hdr_size] (starting 64bit aligned) */
2240
51719889ffcf Record size is allowed to change between index files. This will allow adding
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
82 };
2114
4aedcb4c51cb Added support for per-index sized mail_index_record.
Timo Sirainen <tss@iki.fi>
parents: 2073
diff changeset
83
3016
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
84 struct mail_index_keyword_header {
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
85 uint32_t keywords_count;
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
86 /* struct mail_index_keyword_header_rec[] */
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
87 /* char name[][] */
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
88 };
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
89
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
90 struct mail_index_keyword_header_rec {
5398
730156248978 cleanup
Timo Sirainen <tss@iki.fi>
parents: 5022
diff changeset
91 uint32_t unused; /* for backwards compatibility */
3016
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
92 uint32_t name_offset; /* relative to beginning of name[] */
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
93 };
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
94
3816
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
95 enum mail_index_sync_handler_type {
4283
22b54c007b91 We were updating index->map also when syncing views.
Timo Sirainen <tss@iki.fi>
parents: 4244
diff changeset
96 MAIL_INDEX_SYNC_HANDLER_FILE = 0x01,
22b54c007b91 We were updating index->map also when syncing views.
Timo Sirainen <tss@iki.fi>
parents: 4244
diff changeset
97 MAIL_INDEX_SYNC_HANDLER_HEAD = 0x02,
22b54c007b91 We were updating index->map also when syncing views.
Timo Sirainen <tss@iki.fi>
parents: 4244
diff changeset
98 MAIL_INDEX_SYNC_HANDLER_VIEW = 0x04
3816
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
99 };
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
100
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
101 struct mail_index_sync_handler {
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
102 mail_index_sync_handler_t *callback;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
103 enum mail_index_sync_handler_type type;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
104 };
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
105
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
106 struct mail_index_registered_ext {
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
107 const char *name;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
108 uint32_t index_idx; /* index ext_id */
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
109 uint32_t hdr_size; /* size of mail_index_ext_header.data[] */
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
110 uint16_t record_size;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
111 uint16_t record_align;
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
112
3817
dc560ef2654b Moved expunge handlers to mail_index_registered_ext.
Timo Sirainen <tss@iki.fi>
parents: 3816
diff changeset
113 struct mail_index_sync_handler sync_handler;
4490
56d17f2e1d66 Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents: 4453
diff changeset
114 mail_index_expunge_handler_t *expunge_handler;
56d17f2e1d66 Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents: 4453
diff changeset
115
56d17f2e1d66 Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents: 4453
diff changeset
116 void *expunge_context;
56d17f2e1d66 Added support for calling expunge handler for an extension even if the
Timo Sirainen <tss@iki.fi>
parents: 4453
diff changeset
117 unsigned int expunge_handler_call_always:1;
3816
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
118 };
575c79a0aabc Cleanup: Created mail_index_registered_ext which is used for
Timo Sirainen <tss@iki.fi>
parents: 3805
diff changeset
119
6165
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
120 struct mail_index_record_map {
14920
a097ef0a9d6d Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name
Timo Sirainen <tss@iki.fi>
parents: 14918
diff changeset
121 ARRAY(struct mail_index_map *) maps;
6165
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
122
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
123 void *mmap_base;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
124 size_t mmap_size, mmap_used_size;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
125
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
126 buffer_t *buffer;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
127
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
128 void *records; /* struct mail_index_record[] */
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
129 unsigned int records_count;
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
130
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7367
diff changeset
131 struct mail_index_map_modseq *modseq;
6167
606f3d9f2769 Prevent copying record data when appending new messages.
Timo Sirainen <tss@iki.fi>
parents: 6165
diff changeset
132 uint32_t last_appended_uid;
6165
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
133 };
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
134
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
135 struct mail_index_map {
5853
d9f319a9d6e0 mail_index_map now contains pointer to index, so all functions taking map
Timo Sirainen <tss@iki.fi>
parents: 5820
diff changeset
136 struct mail_index *index;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
137 int refcount;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
138
2884
173a0296e81d Update view's header only after syncing the view. This and other changes fix
Timo Sirainen <tss@iki.fi>
parents: 2869
diff changeset
139 struct mail_index_header hdr;
2563
5752e5ac88f0 Save extra record/header infos into index file permanently.
Timo Sirainen <tss@iki.fi>
parents: 2407
diff changeset
140 const void *hdr_base;
6165
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
141 buffer_t *hdr_copy_buf;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
142
2671
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in
Timo Sirainen <tss@iki.fi>
parents: 2563
diff changeset
143 pool_t extension_pool;
14920
a097ef0a9d6d Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name
Timo Sirainen <tss@iki.fi>
parents: 14918
diff changeset
144 ARRAY(struct mail_index_ext) extensions;
a097ef0a9d6d Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name
Timo Sirainen <tss@iki.fi>
parents: 14918
diff changeset
145 ARRAY(uint32_t) ext_id_map; /* index -> file */
2240
51719889ffcf Record size is allowed to change between index files. This will allow adding
Timo Sirainen <tss@iki.fi>
parents: 2201
diff changeset
146
14920
a097ef0a9d6d Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name
Timo Sirainen <tss@iki.fi>
parents: 14918
diff changeset
147 ARRAY(unsigned int) keyword_idx_map; /* file -> index */
3016
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
148
6165
6418139447fa Created a new struct mail_index_record_map which can be shared by multiple
Timo Sirainen <tss@iki.fi>
parents: 5972
diff changeset
149 struct mail_index_record_map *rec_map;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
150 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
151
5453
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
152 struct mail_index_module_register {
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
153 unsigned int id;
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
154 };
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
155
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
156 union mail_index_module_context {
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
157 struct mail_index_module_register *reg;
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
158 };
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
159
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
160 struct mail_index {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
161 char *dir, *prefix;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
162
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
163 struct mail_cache *cache;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
164 struct mail_transaction_log *log;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
165
10677
93fe3aa23bdb lib-index: Don't close index files if they're still being used by another mailbox.
Timo Sirainen <tss@iki.fi>
parents: 10673
diff changeset
166 unsigned int open_count;
9696
126132cb1c19 index: Removed duplication of mail_index_open_flags from struct mail_index.
Timo Sirainen <tss@iki.fi>
parents: 9612
diff changeset
167 enum mail_index_open_flags flags;
11634
7f19062f58fd Renamed fsync_disable to mail_fsync=optimized|always|never.
Timo Sirainen <tss@iki.fi>
parents: 11623
diff changeset
168 enum fsync_mode fsync_mode;
15063
0efc12f3eb2a lib-index: Changed mail_index_set_fsync_mode() to use a separate enum for the mask.
Timo Sirainen <tss@iki.fi>
parents: 14923
diff changeset
169 enum mail_index_fsync_mask fsync_mask;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
170 mode_t mode;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
171 gid_t gid;
9537
b373de4973cd Whenever file's group changing fails, show the group origin in the error message.
Timo Sirainen <tss@iki.fi>
parents: 9420
diff changeset
172 char *gid_origin;
2115
f720b3f15333 Modifying extra_records should work now.
Timo Sirainen <tss@iki.fi>
parents: 2114
diff changeset
173
21205
5d905618c4b1 lib-index: Add mail_index_set_log_rotation()
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21185
diff changeset
174 uoff_t log_rotate_min_size, log_rotate_max_size;
5d905618c4b1 lib-index: Add mail_index_set_log_rotation()
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21185
diff changeset
175 unsigned int log_rotate_min_created_ago_secs;
5d905618c4b1 lib-index: Add mail_index_set_log_rotation()
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21185
diff changeset
176 unsigned int log_rotate_log2_stale_secs;
22326
ba1a35c5ead7 lib-index: Track .log.2 rotation time in index header
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21205
diff changeset
177 uint32_t pending_log2_rotate_time;
21205
5d905618c4b1 lib-index: Add mail_index_set_log_rotation()
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21185
diff changeset
178
2671
3b70ddb51771 Renamed "extra record info" and variations of it to "extension" or "ext" in
Timo Sirainen <tss@iki.fi>
parents: 2563
diff changeset
179 pool_t extension_pool;
14920
a097ef0a9d6d Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name
Timo Sirainen <tss@iki.fi>
parents: 14918
diff changeset
180 ARRAY(struct mail_index_registered_ext) extensions;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
181
14079
2ab26bb55346 lib-index: Added mail_index_set_ext_init_data() for adding data to index on creation.
Timo Sirainen <tss@iki.fi>
parents: 13806
diff changeset
182 uint32_t ext_hdr_init_id;
2ab26bb55346 lib-index: Added mail_index_set_ext_init_data() for adding data to index on creation.
Timo Sirainen <tss@iki.fi>
parents: 13806
diff changeset
183 void *ext_hdr_init_data;
2ab26bb55346 lib-index: Added mail_index_set_ext_init_data() for adding data to index on creation.
Timo Sirainen <tss@iki.fi>
parents: 13806
diff changeset
184
14920
a097ef0a9d6d Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name
Timo Sirainen <tss@iki.fi>
parents: 14918
diff changeset
185 ARRAY(mail_index_sync_lost_handler_t *) sync_lost_handlers;
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
186
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
187 char *filepath;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
188 int fd;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
189
5689
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
190 struct mail_index_map *map;
21185
47e4d310445b lib-index: Limit mmap syscall errors in index to 1/s
Aki Tuomi <aki.tuomi@dovecot.fi>
parents: 21176
diff changeset
191
47e4d310445b lib-index: Limit mmap syscall errors in index to 1/s
Aki Tuomi <aki.tuomi@dovecot.fi>
parents: 21176
diff changeset
192 time_t last_mmap_error_time;
47e4d310445b lib-index: Limit mmap syscall errors in index to 1/s
Aki Tuomi <aki.tuomi@dovecot.fi>
parents: 21176
diff changeset
193
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
194 uint32_t indexid;
6377
29f427039e00 fsck won't fail anymore with "corrupted index", all problems are fixed.
Timo Sirainen <tss@iki.fi>
parents: 6375
diff changeset
195 unsigned int inconsistency_id;
29f427039e00 fsck won't fail anymore with "corrupted index", all problems are fixed.
Timo Sirainen <tss@iki.fi>
parents: 6375
diff changeset
196
5784
9493c7f1ebca Updating index file by overwriting changed parts didn't work correctly.
Timo Sirainen <tss@iki.fi>
parents: 5775
diff changeset
197 /* last_read_log_file_* contains the seq/offsets we last read from
5820
e59892590a02 Handle losing index file better. Handle fsck better.
Timo Sirainen <tss@iki.fi>
parents: 5784
diff changeset
198 the main index file's headers. these are used to figure out when
19284
ef458c0a98b1 lib-index: Removed some unnecessary fields.
Timo Sirainen <tss@iki.fi>
parents: 18627
diff changeset
199 the main index file should be updated. */
5784
9493c7f1ebca Updating index file by overwriting changed parts didn't work correctly.
Timo Sirainen <tss@iki.fi>
parents: 5775
diff changeset
200 uint32_t last_read_log_file_seq;
5749
420a386fa27a int/ext/mailbox sync offset changes: Combined mailbox and int offsets to
Timo Sirainen <tss@iki.fi>
parents: 5689
diff changeset
201 uint32_t last_read_log_file_tail_offset;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
202
5820
e59892590a02 Handle losing index file better. Handle fsck better.
Timo Sirainen <tss@iki.fi>
parents: 5784
diff changeset
203 /* transaction log head seq/offset when we last fscked */
e59892590a02 Handle losing index file better. Handle fsck better.
Timo Sirainen <tss@iki.fi>
parents: 5784
diff changeset
204 uint32_t fsck_log_head_file_seq;
e59892590a02 Handle losing index file better. Handle fsck better.
Timo Sirainen <tss@iki.fi>
parents: 5784
diff changeset
205 uoff_t fsck_log_head_file_offset;
e59892590a02 Handle losing index file better. Handle fsck better.
Timo Sirainen <tss@iki.fi>
parents: 5784
diff changeset
206
10367
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored.
Timo Sirainen <tss@iki.fi>
parents: 9696
diff changeset
207 /* syncing will update this if non-NULL */
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored.
Timo Sirainen <tss@iki.fi>
parents: 9696
diff changeset
208 struct mail_index_transaction_commit_result *sync_commit_result;
75d3d4374dda Transaction commits can now track how many uid/modseq updates were ignored.
Timo Sirainen <tss@iki.fi>
parents: 9696
diff changeset
209
4876
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's
Timo Sirainen <tss@iki.fi>
parents: 4662
diff changeset
210 enum file_lock_method lock_method;
11623
c3ee4421e86a Added mail_max_lock_timeout setting.
Timo Sirainen <tss@iki.fi>
parents: 10677
diff changeset
211 unsigned int max_lock_timeout_secs;
4876
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's
Timo Sirainen <tss@iki.fi>
parents: 4662
diff changeset
212
3086
66c44404f9a1 Keyword fixes.
Timo Sirainen <tss@iki.fi>
parents: 3016
diff changeset
213 pool_t keywords_pool;
4451
1a35d53c18fc Array API redesigned to work using unions. It now provides type safety
Timo Sirainen <tss@iki.fi>
parents: 4283
diff changeset
214 ARRAY_TYPE(keywords) keywords;
14923
96fd2c3bf932 Reverted "support for non-pointers" part of the hash table API changes.
Timo Sirainen <tss@iki.fi>
parents: 14920
diff changeset
215 HASH_TABLE(char *, void *) keywords_hash; /* name -> unsigned int idx */
3086
66c44404f9a1 Keyword fixes.
Timo Sirainen <tss@iki.fi>
parents: 3016
diff changeset
216
3016
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
217 uint32_t keywords_ext_id;
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7367
diff changeset
218 uint32_t modseq_ext_id;
1928
5ca676060080 when growing index file, do it exponentially for this session (files being
Timo Sirainen <tss@iki.fi>
parents: 1925
diff changeset
219
17108
3f352534af56 lib-index: Keep track of views and transactions in linked lists.
Timo Sirainen <tss@iki.fi>
parents: 16532
diff changeset
220 struct mail_index_view *views;
12762
74300385cce0 lib-index: Give better assert-crashes if view or log view isn't closed.
Timo Sirainen <tss@iki.fi>
parents: 12249
diff changeset
221
5453
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
222 /* Module-specific contexts. */
14920
a097ef0a9d6d Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name
Timo Sirainen <tss@iki.fi>
parents: 14918
diff changeset
223 ARRAY(union mail_index_module_context *) module_contexts;
4451
1a35d53c18fc Array API redesigned to work using unions. It now provides type safety
Timo Sirainen <tss@iki.fi>
parents: 4283
diff changeset
224
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
225 char *error;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
226 unsigned int nodiskspace:1;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
227 unsigned int index_lock_timeout:1;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
228
10673
6552652a9504 lib-index: Index deletion is now a request that gets finalized by index sync.
Timo Sirainen <tss@iki.fi>
parents: 10655
diff changeset
229 unsigned int index_delete_requested:1; /* next sync sets it deleted */
10655
fc0ac73f0b36 Added support for marking mailbox index deleted. Don't allow any changes after that.
Timo Sirainen <tss@iki.fi>
parents: 10367
diff changeset
230 unsigned int index_deleted:1; /* no changes allowed anymore */
12249
41e99ee5c1dd lib-index: Avoid unnecessary pread()s at end of transaction log file.
Timo Sirainen <tss@iki.fi>
parents: 11634
diff changeset
231 unsigned int log_sync_locked:1;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
232 unsigned int readonly:1;
3179
c620532e7bc8 Assert if mail_index_map() is called recursively.
Timo Sirainen <tss@iki.fi>
parents: 3177
diff changeset
233 unsigned int mapping:1;
6756
e78c51d6305f Fixes to handling cache file syncs and expunging.
Timo Sirainen <tss@iki.fi>
parents: 6736
diff changeset
234 unsigned int syncing:1;
6331
081fd0acddd2 If we notice that dovecot.index.log gets deleted, make sure that
Timo Sirainen <tss@iki.fi>
parents: 6275
diff changeset
235 unsigned int need_recreate:1;
8993
48d2377939c2 Write to main index file less often.
Timo Sirainen <tss@iki.fi>
parents: 8973
diff changeset
236 unsigned int index_min_write:1;
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7367
diff changeset
237 unsigned int modseqs_enabled:1;
7573
de08cc81da73 Fixes to handling races in initial index creation.
Timo Sirainen <tss@iki.fi>
parents: 7367
diff changeset
238 unsigned int initial_create:1;
8541
ca3dd777b405 mmap_disable=yes: Opening index always tried to read it from transaction log.
Timo Sirainen <tss@iki.fi>
parents: 8146
diff changeset
239 unsigned int initial_mapped:1;
13806
8535def4ad01 lib-index: Added mail_index_reset_fscked()
Timo Sirainen <tss@iki.fi>
parents: 12762
diff changeset
240 unsigned int fscked:1;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
241 };
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
242
5453
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
243 extern struct mail_index_module_register mail_index_module_register;
a0a1432bb4e8 Better type safety to module_contexts arrays. Already fixed some bugs.
Timo Sirainen <tss@iki.fi>
parents: 5398
diff changeset
244
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
245 /* Add/replace sync handler for specified extra record. */
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
246 void mail_index_register_expunge_handler(struct mail_index *index,
4903
204d7edc7cdc Added context parameter type safety checks for most callback APIs.
Timo Sirainen <tss@iki.fi>
parents: 4876
diff changeset
247 uint32_t ext_id, bool call_always,
204d7edc7cdc Added context parameter type safety checks for most callback APIs.
Timo Sirainen <tss@iki.fi>
parents: 4876
diff changeset
248 mail_index_expunge_handler_t *callback,
204d7edc7cdc Added context parameter type safety checks for most callback APIs.
Timo Sirainen <tss@iki.fi>
parents: 4876
diff changeset
249 void *context);
3224
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
250 void mail_index_unregister_expunge_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
251 uint32_t ext_id);
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
252 void mail_index_register_sync_handler(struct mail_index *index, uint32_t ext_id,
2869
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well.
Timo Sirainen <tss@iki.fi>
parents: 2853
diff changeset
253 mail_index_sync_handler_t *cb,
d2daf90c79f1 Cache invalidating needed to hook into view syncing as well.
Timo Sirainen <tss@iki.fi>
parents: 2853
diff changeset
254 enum mail_index_sync_handler_type type);
3224
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
255 void mail_index_unregister_sync_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
256 uint32_t ext_id);
2934
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index
Timo Sirainen <tss@iki.fi>
parents: 2921
diff changeset
257 void mail_index_register_sync_lost_handler(struct mail_index *index,
c3ae75597952 Added "sync lost" handlers which are called with mmap_disable=yes when index
Timo Sirainen <tss@iki.fi>
parents: 2921
diff changeset
258 mail_index_sync_lost_handler_t *cb);
3224
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
259 void mail_index_unregister_sync_lost_handler(struct mail_index *index,
f6e0d1826645 Added sync/sync_lost/expunge handler unregistering, and also do that. Fixes
Timo Sirainen <tss@iki.fi>
parents: 3191
diff changeset
260 mail_index_sync_lost_handler_t *cb);
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
261
17904
0a34975b2dcc lib-index: Added path parameter to mail_index_create_tmp_file()
Timo Sirainen <tss@iki.fi>
parents: 17427
diff changeset
262 int mail_index_create_tmp_file(struct mail_index *index,
0a34975b2dcc lib-index: Added path parameter to mail_index_create_tmp_file()
Timo Sirainen <tss@iki.fi>
parents: 17427
diff changeset
263 const char *path_prefix, const char **path_r);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
264
5689
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
265 int mail_index_try_open_only(struct mail_index *index);
6390
ff48b008150f Fix ESTALE handling when reading main index.
Timo Sirainen <tss@iki.fi>
parents: 6386
diff changeset
266 void mail_index_close_file(struct mail_index *index);
21087
3b909cd1264b lib-index: Improve error messages when transaction log is unexpectedly lost
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20716
diff changeset
267 int mail_index_reopen_if_changed(struct mail_index *index,
3b909cd1264b lib-index: Improve error messages when transaction log is unexpectedly lost
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20716
diff changeset
268 const char **reason_r);
5753
929cc98ac04e Moved mail_index_write() to its own file.
Timo Sirainen <tss@iki.fi>
parents: 5749
diff changeset
269 /* Update/rewrite the main index file from index->map */
929cc98ac04e Moved mail_index_write() to its own file.
Timo Sirainen <tss@iki.fi>
parents: 5749
diff changeset
270 void mail_index_write(struct mail_index *index, bool want_rotate);
5689
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
271
6829
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently.
Timo Sirainen <tss@iki.fi>
parents: 6756
diff changeset
272 void mail_index_flush_read_cache(struct mail_index *index, const char *path,
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently.
Timo Sirainen <tss@iki.fi>
parents: 6756
diff changeset
273 int fd, bool locked);
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently.
Timo Sirainen <tss@iki.fi>
parents: 6756
diff changeset
274
2941
89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file.
Timo Sirainen <tss@iki.fi>
parents: 2936
diff changeset
275 int mail_index_lock_fd(struct mail_index *index, const char *path, int fd,
4876
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's
Timo Sirainen <tss@iki.fi>
parents: 4662
diff changeset
276 int lock_type, unsigned int timeout_secs,
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's
Timo Sirainen <tss@iki.fi>
parents: 4662
diff changeset
277 struct file_lock **lock_r);
2723
12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use
Timo Sirainen <tss@iki.fi>
parents: 2683
diff changeset
278
5689
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
279 /* Allocate a new empty map. */
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
280 struct mail_index_map *mail_index_map_alloc(struct mail_index *index);
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
281 /* Replace index->map with the latest index changes. This may reopen the index
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
282 file and/or it may read the latest changes from transaction log. The log is
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
283 read up to EOF, but non-synced expunges are skipped.
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
284
5854
eca7e29dce0d Rewrite index lock handling.
Timo Sirainen <tss@iki.fi>
parents: 5853
diff changeset
285 If we mmap()ed the index file, the map is returned locked.
2893
fd431866c674 mail_index_refresh() isn't public anymore, mail_index_view_open_locked()
Timo Sirainen <tss@iki.fi>
parents: 2885
diff changeset
286
6377
29f427039e00 fsck won't fail anymore with "corrupted index", all problems are fixed.
Timo Sirainen <tss@iki.fi>
parents: 6375
diff changeset
287 Returns 1 = ok, 0 = corrupted, -1 = error. */
5689
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
288 int mail_index_map(struct mail_index *index,
5854
eca7e29dce0d Rewrite index lock handling.
Timo Sirainen <tss@iki.fi>
parents: 5853
diff changeset
289 enum mail_index_sync_handler_type type);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
290 /* Unreference given mapping and unmap it if it's dropped to zero. */
5853
d9f319a9d6e0 mail_index_map now contains pointer to index, so all functions taking map
Timo Sirainen <tss@iki.fi>
parents: 5820
diff changeset
291 void mail_index_unmap(struct mail_index_map **map);
5854
eca7e29dce0d Rewrite index lock handling.
Timo Sirainen <tss@iki.fi>
parents: 5853
diff changeset
292
eca7e29dce0d Rewrite index lock handling.
Timo Sirainen <tss@iki.fi>
parents: 5853
diff changeset
293 /* Clone a map. The returned map is always in memory. */
5689
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
294 struct mail_index_map *mail_index_map_clone(const struct mail_index_map *map);
6167
606f3d9f2769 Prevent copying record data when appending new messages.
Timo Sirainen <tss@iki.fi>
parents: 6165
diff changeset
295 void mail_index_record_map_move_to_private(struct mail_index_map *map);
5854
eca7e29dce0d Rewrite index lock handling.
Timo Sirainen <tss@iki.fi>
parents: 5853
diff changeset
296 /* Move a mmaped map to memory. */
5689
c2362f144f15 Initial commit for major index file code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 5686
diff changeset
297 void mail_index_map_move_to_memory(struct mail_index_map *map);
9420
c496f137c843 index: Handle better errors where a new file's group can't be changed.
Timo Sirainen <tss@iki.fi>
parents: 9333
diff changeset
298 void mail_index_fchown(struct mail_index *index, int fd, const char *path);
2683
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
299
6348
ef1c7b2acc10 Only index extension indexes are now called "ext_id". For map extension
Timo Sirainen <tss@iki.fi>
parents: 6342
diff changeset
300 bool mail_index_map_lookup_ext(struct mail_index_map *map, const char *name,
ef1c7b2acc10 Only index extension indexes are now called "ext_id". For map extension
Timo Sirainen <tss@iki.fi>
parents: 6342
diff changeset
301 uint32_t *idx_r);
2683
d41c42a7bf1e Index extensions can now specify record alignment, and it's possible to
Timo Sirainen <tss@iki.fi>
parents: 2671
diff changeset
302 uint32_t
6872
082526b2702e Verify extension fields are correct when registering it from transaction
Timo Sirainen <tss@iki.fi>
parents: 6829
diff changeset
303 mail_index_map_register_ext(struct mail_index_map *map,
082526b2702e Verify extension fields are correct when registering it from transaction
Timo Sirainen <tss@iki.fi>
parents: 6829
diff changeset
304 const char *name, uint32_t ext_offset,
082526b2702e Verify extension fields are correct when registering it from transaction
Timo Sirainen <tss@iki.fi>
parents: 6829
diff changeset
305 const struct mail_index_ext_header *ext_hdr);
6168
28d1cfbbce6e mail_index_map_get_ext_idx() should return bool, not int.
Timo Sirainen <tss@iki.fi>
parents: 6167
diff changeset
306 bool mail_index_map_get_ext_idx(struct mail_index_map *map,
28d1cfbbce6e mail_index_map_get_ext_idx() should return bool, not int.
Timo Sirainen <tss@iki.fi>
parents: 6167
diff changeset
307 uint32_t ext_id, uint32_t *idx_r);
2853
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
308 const struct mail_index_ext *
512dd7d76cdc Removed cache_offset from mail_index_record and changed it to use extension
Timo Sirainen <tss@iki.fi>
parents: 2819
diff changeset
309 mail_index_view_get_ext(struct mail_index_view *view, uint32_t ext_id);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
310
9333
190669ac816c Created mail_index_map_lookup_seq_range() from code in mail-index-view.c.
Timo Sirainen <tss@iki.fi>
parents: 9331
diff changeset
311 void mail_index_map_lookup_seq_range(struct mail_index_map *map,
190669ac816c Created mail_index_map_lookup_seq_range() from code in mail-index-view.c.
Timo Sirainen <tss@iki.fi>
parents: 9331
diff changeset
312 uint32_t first_uid, uint32_t last_uid,
190669ac816c Created mail_index_map_lookup_seq_range() from code in mail-index-view.c.
Timo Sirainen <tss@iki.fi>
parents: 9331
diff changeset
313 uint32_t *first_seq_r,
190669ac816c Created mail_index_map_lookup_seq_range() from code in mail-index-view.c.
Timo Sirainen <tss@iki.fi>
parents: 9331
diff changeset
314 uint32_t *last_seq_r);
190669ac816c Created mail_index_map_lookup_seq_range() from code in mail-index-view.c.
Timo Sirainen <tss@iki.fi>
parents: 9331
diff changeset
315
18627
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why.
Timo Sirainen <tss@iki.fi>
parents: 17931
diff changeset
316 /* Returns 1 on success, 0 on non-critical errors we want to silently fix,
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why.
Timo Sirainen <tss@iki.fi>
parents: 17931
diff changeset
317 -1 if map isn't usable. The caller is responsible for logging the errors
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why.
Timo Sirainen <tss@iki.fi>
parents: 17931
diff changeset
318 if -1 is returned. */
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why.
Timo Sirainen <tss@iki.fi>
parents: 17931
diff changeset
319 int mail_index_map_check_header(struct mail_index_map *map,
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why.
Timo Sirainen <tss@iki.fi>
parents: 17931
diff changeset
320 const char **error_r);
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why.
Timo Sirainen <tss@iki.fi>
parents: 17931
diff changeset
321 /* Returns 1 if header is usable, 0 or -1 if not. The caller should log an
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why.
Timo Sirainen <tss@iki.fi>
parents: 17931
diff changeset
322 error if -1 is returned, but not if 0 is returned. */
9331
768b25cabd6c Moved some code from mail-index-map.c to mail-index-map-hdr.c
Timo Sirainen <tss@iki.fi>
parents: 9001
diff changeset
323 bool mail_index_check_header_compat(struct mail_index *index,
768b25cabd6c Moved some code from mail-index-map.c to mail-index-map-hdr.c
Timo Sirainen <tss@iki.fi>
parents: 9001
diff changeset
324 const struct mail_index_header *hdr,
18627
69630e6048fd lib-index: If header is corrupted after syncing, log the reason why.
Timo Sirainen <tss@iki.fi>
parents: 17931
diff changeset
325 uoff_t file_size, const char **error_r);
9331
768b25cabd6c Moved some code from mail-index-map.c to mail-index-map-hdr.c
Timo Sirainen <tss@iki.fi>
parents: 9001
diff changeset
326 int mail_index_map_parse_extensions(struct mail_index_map *map);
5853
d9f319a9d6e0 mail_index_map now contains pointer to index, so all functions taking map
Timo Sirainen <tss@iki.fi>
parents: 5820
diff changeset
327 int mail_index_map_parse_keywords(struct mail_index_map *map);
3016
61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet.
Timo Sirainen <tss@iki.fi>
parents: 2941
diff changeset
328
9331
768b25cabd6c Moved some code from mail-index-map.c to mail-index-map-hdr.c
Timo Sirainen <tss@iki.fi>
parents: 9001
diff changeset
329 void mail_index_map_init_extbufs(struct mail_index_map *map,
768b25cabd6c Moved some code from mail-index-map.c to mail-index-map-hdr.c
Timo Sirainen <tss@iki.fi>
parents: 9001
diff changeset
330 unsigned int initial_count);
6385
9dde743dfbc1 If extension header is broken, drop it when fscking.
Timo Sirainen <tss@iki.fi>
parents: 6377
diff changeset
331 int mail_index_map_ext_get_next(struct mail_index_map *map,
9dde743dfbc1 If extension header is broken, drop it when fscking.
Timo Sirainen <tss@iki.fi>
parents: 6377
diff changeset
332 unsigned int *offset,
9dde743dfbc1 If extension header is broken, drop it when fscking.
Timo Sirainen <tss@iki.fi>
parents: 6377
diff changeset
333 const struct mail_index_ext_header **ext_hdr_r,
9dde743dfbc1 If extension header is broken, drop it when fscking.
Timo Sirainen <tss@iki.fi>
parents: 6377
diff changeset
334 const char **name_r);
9dde743dfbc1 If extension header is broken, drop it when fscking.
Timo Sirainen <tss@iki.fi>
parents: 6377
diff changeset
335 int mail_index_map_ext_hdr_check(const struct mail_index_header *hdr,
9dde743dfbc1 If extension header is broken, drop it when fscking.
Timo Sirainen <tss@iki.fi>
parents: 6377
diff changeset
336 const struct mail_index_ext_header *ext_hdr,
9dde743dfbc1 If extension header is broken, drop it when fscking.
Timo Sirainen <tss@iki.fi>
parents: 6377
diff changeset
337 const char *name, const char **error_r);
6386
1d993d28e12f fsck fixes now keyword header problems
Timo Sirainen <tss@iki.fi>
parents: 6385
diff changeset
338 unsigned int mail_index_map_ext_hdr_offset(unsigned int name_len);
6385
9dde743dfbc1 If extension header is broken, drop it when fscking.
Timo Sirainen <tss@iki.fi>
parents: 6377
diff changeset
339
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
340 void mail_index_view_transaction_ref(struct mail_index_view *view);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
341 void mail_index_view_transaction_unref(struct mail_index_view *view);
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
342
6377
29f427039e00 fsck won't fail anymore with "corrupted index", all problems are fixed.
Timo Sirainen <tss@iki.fi>
parents: 6375
diff changeset
343 void mail_index_fsck_locked(struct mail_index *index);
29f427039e00 fsck won't fail anymore with "corrupted index", all problems are fixed.
Timo Sirainen <tss@iki.fi>
parents: 6375
diff changeset
344
14682
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14599
diff changeset
345 void mail_index_set_error(struct mail_index *index, const char *fmt, ...)
6411
6a64e64fa3a3 Renamed __attr_*__ to ATTR_*. Renamed __attrs_used__ to ATTRS_DEFINED.
Timo Sirainen <tss@iki.fi>
parents: 6410
diff changeset
346 ATTR_FORMAT(2, 3);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
347 /* "%s failed with index file %s: %m" */
14682
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14599
diff changeset
348 void mail_index_set_syscall_error(struct mail_index *index,
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14599
diff changeset
349 const char *function);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
350 /* "%s failed with file %s: %m" */
14682
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14599
diff changeset
351 void mail_index_file_set_syscall_error(struct mail_index *index,
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14599
diff changeset
352 const char *filepath,
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14599
diff changeset
353 const char *function);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
354
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
355 #endif