Mercurial > dovecot > core-2.2

annotate src/lib-storage/index/index-search.c @ 22511:f0694e6eda8d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
author Stephan Bosch <stephan.bosch@dovecot.fi>
date Tue, 23 May 2017 14:05:02 +0200
parents fca31317b56c
children a91a473c65f7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
21390
2e2563132d5f Updated copyright notices to include the year 2017.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 21389
diff changeset
1 /* Copyright (c) 2002-2017 Dovecot authors, see the included COPYING file */
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
2
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
3 #include "lib.h"
4949
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
4 #include "ioloop.h"
4607
71b3570946f8 Moved some search parameters to public struct mail_search_context and added
Timo Sirainen <tss@iki.fi>
parents: 4439
diff changeset
5 #include "array.h"
764
f57c52738f90 Renamed IBuffer and OBuffer to IStream and OStream which describes their
Timo Sirainen <tss@iki.fi>
parents: 761
diff changeset
6 #include "istream.h"
7279
ef13dd179331 More fixes to SEARCH BEFORE/ON/SINCE timezone handling.
Timo Sirainen <tss@iki.fi>
parents: 7269
diff changeset
7 #include "utc-offset.h"
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
8 #include "str.h"
9769
8716936443db Moved all struct timeval comparing/calculation code to lib/time-util.
Timo Sirainen <tss@iki.fi>
parents: 9704
diff changeset
9 #include "time-util.h"
13927
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
10 #include "unichar.h"
11197
3c9770bae39e lib-storage: Added MAILBOX_GLOB search arg. Query builders now use it instead of MAILBOX.
Timo Sirainen <tss@iki.fi>
parents: 11141
diff changeset
11 #include "imap-match.h"
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
12 #include "message-address.h"
903
fd8888f6f037 Naming style changes, finally got tired of most of the typedefs. Also the
Timo Sirainen <tss@iki.fi>
parents: 898
diff changeset
13 #include "message-date.h"
5525
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
14 #include "message-search.h"
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
15 #include "message-parser.h"
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
16 #include "mail-index-modseq.h"
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
17 #include "index-storage.h"
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
18 #include "index-mail.h"
4303
5f03738219a6 Changed mail-storage API to do the mail sorting internally. Optimized it
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4259
diff changeset
19 #include "index-sort.h"
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
20 #include "mail-search.h"
7647
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
21 #include "mailbox-search-result-private.h"
12608
b96efbad2fa4 lib-storage: Index search context is now more accessible by backends.
Timo Sirainen <tss@iki.fi>
parents: 12550
diff changeset
22 #include "index-search-private.h"
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
23
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
24 #include <ctype.h>
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
25
4949
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
26 #define SEARCH_NOTIFY_INTERVAL_SECS 10
4939
ff2272c228cc Dovecot is now able to execute multiple commands at the same time.
Timo Sirainen <tss@iki.fi>
parents: 4907
diff changeset
27
9164
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
28 #define SEARCH_COST_DENTRY 3ULL
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
29 #define SEARCH_COST_ATTR 1ULL
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
30 #define SEARCH_COST_FILES_READ 25ULL
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
31 #define SEARCH_COST_KBYTE 15ULL
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
32 #define SEARCH_COST_CACHE 1ULL
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
33
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
34 #define SEARCH_MIN_NONBLOCK_USECS 200000
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
35 #define SEARCH_MAX_NONBLOCK_USECS 250000
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
36 #define SEARCH_INITIAL_MAX_COST 30000
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
37 #define SEARCH_RECALC_MIN_USECS 50000
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
38
903
fd8888f6f037 Naming style changes, finally got tired of most of the typedefs. Also the
Timo Sirainen <tss@iki.fi>
parents: 898
diff changeset
39 struct search_header_context {
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
40 struct index_search_context *index_ctx;
12871
d399d5c65aca Changed header/body searching to do search key charset translation earlier.
Timo Sirainen <tss@iki.fi>
parents: 12870
diff changeset
41 struct index_mail *imail;
903
fd8888f6f037 Naming style changes, finally got tired of most of the typedefs. Also the
Timo Sirainen <tss@iki.fi>
parents: 898
diff changeset
42 struct mail_search_arg *args;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
43
19757
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
44 struct message_block decoded_block;
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
45 bool decoded_block_set;
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
46
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
47 struct message_header_line *hdr;
924
4f697dde0fca THREAD=REFERENCES implementation. Doesn't crash, but I'm not sure how
Timo Sirainen <tss@iki.fi>
parents: 903
diff changeset
48
3325
38eaa792c405 Don't call index_mail_parse_header() twice when searching headers. Caused
Timo Sirainen <tss@iki.fi>
parents: 3301
diff changeset
49 unsigned int parse_headers:1;
924
4f697dde0fca THREAD=REFERENCES implementation. Doesn't crash, but I'm not sure how
Timo Sirainen <tss@iki.fi>
parents: 903
diff changeset
50 unsigned int custom_header:1;
4f697dde0fca THREAD=REFERENCES implementation. Doesn't crash, but I'm not sure how
Timo Sirainen <tss@iki.fi>
parents: 903
diff changeset
51 unsigned int threading:1;
903
fd8888f6f037 Naming style changes, finally got tired of most of the typedefs. Also the
Timo Sirainen <tss@iki.fi>
parents: 898
diff changeset
52 };
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
53
903
fd8888f6f037 Naming style changes, finally got tired of most of the typedefs. Also the
Timo Sirainen <tss@iki.fi>
parents: 898
diff changeset
54 struct search_body_context {
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
55 struct index_search_context *index_ctx;
903
fd8888f6f037 Naming style changes, finally got tired of most of the typedefs. Also the
Timo Sirainen <tss@iki.fi>
parents: 898
diff changeset
56 struct istream *input;
11000
679e99e430d7 lib-storage: mail_get_parts() no longer returns const pointer.
Timo Sirainen <tss@iki.fi>
parents: 10946
diff changeset
57 struct message_part *part;
903
fd8888f6f037 Naming style changes, finally got tired of most of the typedefs. Also the
Timo Sirainen <tss@iki.fi>
parents: 898
diff changeset
58 };
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
59
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
60 static void search_parse_msgset_args(unsigned int messages_count,
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
61 struct mail_search_arg *args,
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
62 uint32_t *seq1_r, uint32_t *seq2_r);
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
63
14629
c93ca5e46a8a Marked functions parameters that are allowed to be NULL. Some APIs were also changed.
Timo Sirainen <tss@iki.fi>
parents: 14319
diff changeset
64 static void ATTR_NULL(2)
14921
d3db2ba15d00 Removed CONTEXT_TYPE_SAFETY macro and reimplemented its functionality better.
Timo Sirainen <tss@iki.fi>
parents: 14738
diff changeset
65 search_none(struct mail_search_arg *arg ATTR_UNUSED, void *ctx ATTR_UNUSED)
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
66 {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
67 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
68
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
69 static void search_set_failed(struct index_search_context *ctx)
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
70 {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
71 if (ctx->failed)
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
72 return;
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
73
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
74 /* remember the first failure */
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
75 mail_storage_last_error_push(ctx->box->storage);
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
76 ctx->failed = TRUE;
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
77 }
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
78
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
79 static void search_cur_mail_failed(struct index_search_context *ctx)
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
80 {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
81 switch (mailbox_get_last_mail_error(ctx->cur_mail->box)) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
82 case MAIL_ERROR_EXPUNGED:
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
83 ctx->mail_ctx.seen_lost_data = TRUE;
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
84 break;
21606
6f2e7eb1ee90 lib-storage: Add explicit MAIL_ERROR_LOOKUP_ABORT error
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21605
diff changeset
85 case MAIL_ERROR_LOOKUP_ABORTED:
6f2e7eb1ee90 lib-storage: Add explicit MAIL_ERROR_LOOKUP_ABORT error
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21605
diff changeset
86 /* expected failure */
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
87 break;
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
88 default:
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
89 search_set_failed(ctx);
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
90 break;
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
91 }
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
92 }
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
93
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
94 static void search_init_arg(struct mail_search_arg *arg,
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
95 struct index_search_context *ctx)
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
96 {
12550
2c299c0e3bc8 lib-storage: Moved some items from mailbox_get_status() to a new mailbox_get_metadata().
Timo Sirainen <tss@iki.fi>
parents: 12254
diff changeset
97 struct mailbox_metadata metadata;
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
98 bool match;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
99
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
100 switch (arg->type) {
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
101 case SEARCH_SEQSET:
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
102 ctx->have_seqsets = TRUE;
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
103 break;
7618
6dbd70663adf Added support for SEARCH_UIDSET.
Timo Sirainen <tss@iki.fi>
parents: 7414
diff changeset
104 case SEARCH_UIDSET:
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
105 case SEARCH_INTHREAD:
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
106 case SEARCH_FLAGS:
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
107 case SEARCH_KEYWORDS:
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
108 case SEARCH_MODSEQ:
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
109 if (arg->type == SEARCH_MODSEQ)
10660
56b1d4dd9c7d lib-storage: *_mailboxes don't descend from index_mailbox anymore, it's now a context.
Timo Sirainen <tss@iki.fi>
parents: 10657
diff changeset
110 mail_index_modseq_enable(ctx->box->index);
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
111 ctx->have_index_args = TRUE;
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
112 break;
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
113 case SEARCH_MAILBOX_GUID:
12550
2c299c0e3bc8 lib-storage: Moved some items from mailbox_get_status() to a new mailbox_get_metadata().
Timo Sirainen <tss@iki.fi>
parents: 12254
diff changeset
114 if (mailbox_get_metadata(ctx->box, MAILBOX_METADATA_GUID,
2c299c0e3bc8 lib-storage: Moved some items from mailbox_get_status() to a new mailbox_get_metadata().
Timo Sirainen <tss@iki.fi>
parents: 12254
diff changeset
115 &metadata) < 0) {
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
116 /* result will be unknown */
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
117 break;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
118 }
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
119
13287
957060ca5b69 Moved GUID code to liblib. Use guid_128_t type consistently everywhere.
Timo Sirainen <tss@iki.fi>
parents: 13251
diff changeset
120 match = strcmp(guid_128_to_string(metadata.guid),
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
121 arg->value.str) == 0;
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
122 if (match != arg->match_not)
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
123 arg->match_always = TRUE;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
124 else
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
125 arg->nonmatch_always = TRUE;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
126 break;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
127 case SEARCH_MAILBOX:
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
128 case SEARCH_MAILBOX_GLOB:
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
129 ctx->have_mailbox_args = TRUE;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
130 break;
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
131 case SEARCH_ALL:
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
132 if (!arg->match_not)
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
133 arg->match_always = TRUE;
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
134 else
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
135 arg->nonmatch_always = TRUE;
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
136 break;
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
137 default:
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
138 break;
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
139 }
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
140 }
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
141
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
142 static void search_seqset_arg(struct mail_search_arg *arg,
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
143 struct index_search_context *ctx)
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
144 {
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
145 if (arg->type == SEARCH_SEQSET) {
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
146 if (seq_range_exists(&arg->value.seqset, ctx->mail_ctx.seq))
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
147 ARG_SET_RESULT(arg, 1);
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
148 else
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
149 ARG_SET_RESULT(arg, 0);
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
150 }
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
151 }
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
152
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
153 static int search_arg_match_keywords(struct index_search_context *ctx,
6965
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
154 struct mail_search_arg *arg)
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
155 {
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
156 ARRAY_TYPE(keyword_indexes) keyword_indexes_arr;
18722
e6513dd519d5 lib-storage: Cleanup - separate search arg values that are set by mail_search_init()
Timo Sirainen <tss@iki.fi>
parents: 18147
diff changeset
157 const struct mail_keywords *search_kws = arg->initialized.keywords;
6965
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
158 const unsigned int *keyword_indexes;
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
159 unsigned int i, j, count;
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
160
22005
fca31317b56c lib-storage: Fix searching when search query has invalid keywords.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21654
diff changeset
161 if (search_kws->count == 0) {
fca31317b56c lib-storage: Fix searching when search query has invalid keywords.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21654
diff changeset
162 /* invalid keyword - never matches */
fca31317b56c lib-storage: Fix searching when search query has invalid keywords.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21654
diff changeset
163 return 0;
fca31317b56c lib-storage: Fix searching when search query has invalid keywords.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21654
diff changeset
164 }
fca31317b56c lib-storage: Fix searching when search query has invalid keywords.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21654
diff changeset
165
6965
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
166 t_array_init(&keyword_indexes_arr, 128);
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
167 mail_index_lookup_keywords(ctx->view, ctx->mail_ctx.seq,
6965
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
168 &keyword_indexes_arr);
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
169 keyword_indexes = array_get(&keyword_indexes_arr, &count);
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
170
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
171 /* there probably aren't many keywords, so O(n*m) for now */
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
172 for (i = 0; i < search_kws->count; i++) {
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
173 for (j = 0; j < count; j++) {
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
174 if (search_kws->idx[i] == keyword_indexes[j])
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
175 break;
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
176 }
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
177 if (j == count)
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
178 return 0;
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
179 }
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
180 return 1;
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
181 }
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
182
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
183 static bool
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
184 index_search_get_pvt(struct index_search_context *ctx, uint32_t uid)
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
185 {
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
186 index_transaction_init_pvt(ctx->mail_ctx.transaction);
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
187
14711
4fb231804670 lib-storage: Search crashfix.
Timo Sirainen <tss@iki.fi>
parents: 14710
diff changeset
188 if (ctx->pvt_uid == uid)
4fb231804670 lib-storage: Search crashfix.
Timo Sirainen <tss@iki.fi>
parents: 14710
diff changeset
189 return ctx->pvt_seq != 0;
14709
e9e1b97c78c7 lib-storage: Fixed searching message flags on mailboxes without private index.
Timo Sirainen <tss@iki.fi>
parents: 14704
diff changeset
190 ctx->pvt_uid = uid;
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
191 return mail_index_lookup_seq(ctx->mail_ctx.transaction->view_pvt,
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
192 uid, &ctx->pvt_seq);
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
193 }
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
194
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
195 /* Returns >0 = matched, 0 = not matched, -1 = unknown */
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
196 static int search_arg_match_index(struct index_search_context *ctx,
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
197 struct mail_search_arg *arg,
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
198 const struct mail_index_record *rec)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
199 {
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
200 enum mail_flags flags, pvt_flags_mask;
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
201 uint64_t modseq;
6965
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
202 int ret;
1947
777da553d1d3 Recent-flag should work now
Timo Sirainen <tss@iki.fi>
parents: 1917
diff changeset
203
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
204 switch (arg->type) {
7618
6dbd70663adf Added support for SEARCH_UIDSET.
Timo Sirainen <tss@iki.fi>
parents: 7414
diff changeset
205 case SEARCH_UIDSET:
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
206 case SEARCH_INTHREAD:
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
207 return seq_range_exists(&arg->value.seqset, rec->uid);
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
208 case SEARCH_FLAGS:
7426
11e7ec83665d Ignore \Recent flag in indexes when searching them (leftovers from v1.0).
Timo Sirainen <tss@iki.fi>
parents: 7414
diff changeset
209 /* recent flag shouldn't be set, but indexes from v1.0.x
11e7ec83665d Ignore \Recent flag in indexes when searching them (leftovers from v1.0).
Timo Sirainen <tss@iki.fi>
parents: 7414
diff changeset
210 may contain it. */
11e7ec83665d Ignore \Recent flag in indexes when searching them (leftovers from v1.0).
Timo Sirainen <tss@iki.fi>
parents: 7414
diff changeset
211 flags = rec->flags & ~MAIL_RECENT;
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
212 if ((arg->value.flags & MAIL_RECENT) != 0 &&
18975
f78bd5be4e47 lib-storage: Moved most of the \Recent flag handling code to mailbox-recent-flags.c
Timo Sirainen <tss@iki.fi>
parents: 18722
diff changeset
213 mailbox_recent_flags_have_uid(ctx->box, rec->uid))
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
214 flags |= MAIL_RECENT;
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
215 if (ctx->box->view_pvt == NULL) {
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
216 /* no private view (set by view syncing) ->
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
217 no private flags */
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
218 } else {
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
219 pvt_flags_mask = mailbox_get_private_flags_mask(ctx->box);
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
220 flags &= ~pvt_flags_mask;
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
221 if (index_search_get_pvt(ctx, rec->uid)) {
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
222 rec = mail_index_lookup(ctx->mail_ctx.transaction->view_pvt,
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
223 ctx->pvt_seq);
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
224 flags |= rec->flags & pvt_flags_mask;
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
225 }
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
226 }
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
227 return (flags & arg->value.flags) == arg->value.flags;
6965
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
228 case SEARCH_KEYWORDS:
7226
e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents: 7086
diff changeset
229 T_BEGIN {
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
230 ret = search_arg_match_keywords(ctx, arg);
7226
e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and
Timo Sirainen <tss@iki.fi>
parents: 7086
diff changeset
231 } T_END;
6965
ae369569e7de SEARCH_KEYWORDS now uses struct mail_keywords instead of a string array.
Timo Sirainen <tss@iki.fi>
parents: 6962
diff changeset
232 return ret;
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
233 case SEARCH_MODSEQ: {
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
234 if (arg->value.flags != 0) {
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
235 modseq = mail_index_modseq_lookup_flags(ctx->view,
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
236 arg->value.flags, ctx->mail_ctx.seq);
18722
e6513dd519d5 lib-storage: Cleanup - separate search arg values that are set by mail_search_init()
Timo Sirainen <tss@iki.fi>
parents: 18147
diff changeset
237 } else if (arg->initialized.keywords != NULL) {
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
238 modseq = mail_index_modseq_lookup_keywords(ctx->view,
18722
e6513dd519d5 lib-storage: Cleanup - separate search arg values that are set by mail_search_init()
Timo Sirainen <tss@iki.fi>
parents: 18147
diff changeset
239 arg->initialized.keywords, ctx->mail_ctx.seq);
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
240 } else {
7625
98a03cf8ad5d CONDSTORE fixes
Timo Sirainen <tss@iki.fi>
parents: 7620
diff changeset
241 modseq = mail_index_modseq_lookup(ctx->view,
7620
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
242 ctx->mail_ctx.seq);
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
243 }
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
244 return modseq >= arg->value.modseq->modseq;
4b8c1c164d8f Initial CONDSTORE support.
Timo Sirainen <tss@iki.fi>
parents: 7619
diff changeset
245 }
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
246 default:
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
247 return -1;
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
248 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
249 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
250
4907
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
251 static void search_index_arg(struct mail_search_arg *arg,
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
252 struct index_search_context *ctx)
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
253 {
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
254 const struct mail_index_record *rec;
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
255
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
256 rec = mail_index_lookup(ctx->view, ctx->mail_ctx.seq);
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
257 switch (search_arg_match_index(ctx, arg, rec)) {
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
258 case -1:
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
259 /* unknown */
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
260 break;
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
261 case 0:
1069
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
262 ARG_SET_RESULT(arg, 0);
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
263 break;
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
264 default:
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
265 ARG_SET_RESULT(arg, 1);
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
266 break;
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
267 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
268 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
269
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
270 /* Returns >0 = matched, 0 = not matched, -1 = unknown */
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
271 static int search_arg_match_mailbox(struct index_search_context *ctx,
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
272 struct mail_search_arg *arg)
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
273 {
17817
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
274 struct mailbox *box = ctx->cur_mail->box;
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
275 const char *str;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
276
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
277 switch (arg->type) {
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
278 case SEARCH_MAILBOX:
17817
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
279 /* first try to match the mailbox name itself. this is
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
280 important when using "mailbox virtual/foo" parameter foin
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
281 doveadm's search query, otherwise we can never fetch
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
282 anything with doveadm from virtual mailboxes because the
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
283 mailbox parameter is compared to the mail's backend
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
284 mailbox. */
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
285 if (strcmp(box->vname, arg->value.str) == 0)
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
286 return 1;
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
287 if (mail_get_special(ctx->cur_mail, MAIL_FETCH_MAILBOX_NAME,
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
288 &str) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
289 search_cur_mail_failed(ctx);
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
290 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
291 }
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
292
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
293 if (strcasecmp(str, "INBOX") == 0)
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
294 return strcasecmp(arg->value.str, "INBOX") == 0;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
295 return strcmp(str, arg->value.str) == 0;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
296 case SEARCH_MAILBOX_GLOB:
18722
e6513dd519d5 lib-storage: Cleanup - separate search arg values that are set by mail_search_init()
Timo Sirainen <tss@iki.fi>
parents: 18147
diff changeset
297 if (imap_match(arg->initialized.mailbox_glob, box->vname) == IMAP_MATCH_YES)
17817
ad16c74f4f36 lib-storage: SEARCH_MAILBOX* value is now also compared to to the (virtual) mailbox name.
Timo Sirainen <tss@iki.fi>
parents: 17212
diff changeset
298 return 1;
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
299 if (mail_get_special(ctx->cur_mail, MAIL_FETCH_MAILBOX_NAME,
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
300 &str) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
301 search_cur_mail_failed(ctx);
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
302 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
303 }
18722
e6513dd519d5 lib-storage: Cleanup - separate search arg values that are set by mail_search_init()
Timo Sirainen <tss@iki.fi>
parents: 18147
diff changeset
304 return imap_match(arg->initialized.mailbox_glob, str) == IMAP_MATCH_YES;
11238
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
305 default:
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
306 return -1;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
307 }
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
308 }
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
309
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
310 static void search_mailbox_arg(struct mail_search_arg *arg,
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
311 struct index_search_context *ctx)
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
312 {
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
313 switch (search_arg_match_mailbox(ctx, arg)) {
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
314 case -1:
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
315 /* unknown */
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
316 break;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
317 case 0:
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
318 ARG_SET_RESULT(arg, 0);
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
319 break;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
320 default:
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
321 ARG_SET_RESULT(arg, 1);
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
322 break;
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
323 }
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
324 }
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
325
dedf835014a2 lib-storage: Fixes/optimizations to SEARCH_MAILBOX*.
Timo Sirainen <tss@iki.fi>
parents: 11234
diff changeset
326 /* Returns >0 = matched, 0 = not matched, -1 = unknown */
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
327 static int search_arg_match_cached(struct index_search_context *ctx,
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
328 struct mail_search_arg *arg)
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
329 {
7910
c1bbdc2b262e FETCH X-MAILBOX and SEARCH X-MAILBOX can be used with virtual mailboxes to
Timo Sirainen <tss@iki.fi>
parents: 7909
diff changeset
330 const char *str;
7279
ef13dd179331 More fixes to SEARCH BEFORE/ON/SINCE timezone handling.
Timo Sirainen <tss@iki.fi>
parents: 7269
diff changeset
331 struct tm *tm;
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
332 uoff_t virtual_size;
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
333 time_t date;
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
334 int tz_offset;
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
335 bool have_tz_offset;
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
336
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
337 switch (arg->type) {
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
338 /* internal dates */
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
339 case SEARCH_BEFORE:
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
340 case SEARCH_ON:
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
341 case SEARCH_SINCE:
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
342 have_tz_offset = FALSE; tz_offset = 0; date = (time_t)-1;
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
343 switch (arg->value.date_type) {
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
344 case MAIL_SEARCH_DATE_TYPE_SENT:
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
345 if (mail_get_date(ctx->cur_mail, &date, &tz_offset) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
346 search_cur_mail_failed(ctx);
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
347 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
348 }
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
349 have_tz_offset = TRUE;
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
350 break;
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
351 case MAIL_SEARCH_DATE_TYPE_RECEIVED:
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
352 if (mail_get_received_date(ctx->cur_mail, &date) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
353 search_cur_mail_failed(ctx);
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
354 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
355 }
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
356 break;
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
357 case MAIL_SEARCH_DATE_TYPE_SAVED:
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
358 if (mail_get_save_date(ctx->cur_mail, &date) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
359 search_cur_mail_failed(ctx);
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
360 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
361 }
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
362 break;
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
363 }
7279
ef13dd179331 More fixes to SEARCH BEFORE/ON/SINCE timezone handling.
Timo Sirainen <tss@iki.fi>
parents: 7269
diff changeset
364
7640
c78e9204f3f2 Implemented WITHIN extension.
Timo Sirainen <tss@iki.fi>
parents: 7625
diff changeset
365 if ((arg->value.search_flags &
21066
b9a20a0cb54a lib-storage: Reverse MAIL_SEARCH_ARG_FLAG_USE_TZ handling again.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21023
diff changeset
366 MAIL_SEARCH_ARG_FLAG_USE_TZ) == 0) {
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
367 if (!have_tz_offset) {
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
368 tm = localtime(&date);
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
369 tz_offset = utc_offset(tm, date);
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
370 }
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
371 date += tz_offset * 60;
7640
c78e9204f3f2 Implemented WITHIN extension.
Timo Sirainen <tss@iki.fi>
parents: 7625
diff changeset
372 }
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
373
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
374 switch (arg->type) {
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
375 case SEARCH_BEFORE:
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
376 return date < arg->value.time;
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
377 case SEARCH_ON:
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
378 return date >= arg->value.time &&
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
379 date < arg->value.time + 3600*24;
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
380 case SEARCH_SINCE:
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
381 return date >= arg->value.time;
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
382 default:
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
383 /* unreachable */
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
384 break;
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
385 }
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
386
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
387 /* sizes */
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
388 case SEARCH_SMALLER:
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
389 case SEARCH_LARGER:
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
390 if (mail_get_virtual_size(ctx->cur_mail, &virtual_size) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
391 search_cur_mail_failed(ctx);
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
392 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
393 }
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
394
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
395 if (arg->type == SEARCH_SMALLER)
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
396 return virtual_size < arg->value.size;
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
397 else
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
398 return virtual_size > arg->value.size;
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
399
8077
6d51328896d6 Added the concept of Global UIDs that are preserved across copies.
Timo Sirainen <tss@iki.fi>
parents: 8074
diff changeset
400 case SEARCH_GUID:
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
401 if (mail_get_special(ctx->cur_mail, MAIL_FETCH_GUID, &str) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
402 search_cur_mail_failed(ctx);
8077
6d51328896d6 Added the concept of Global UIDs that are preserved across copies.
Timo Sirainen <tss@iki.fi>
parents: 8074
diff changeset
403 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
404 }
8077
6d51328896d6 Added the concept of Global UIDs that are preserved across copies.
Timo Sirainen <tss@iki.fi>
parents: 8074
diff changeset
405 return strcmp(str, arg->value.str) == 0;
17949
98b5ccc539b5 lib-storage: Added X-REAL-UID search parameter.
Timo Sirainen <tss@iki.fi>
parents: 17817
diff changeset
406 case SEARCH_REAL_UID: {
98b5ccc539b5 lib-storage: Added X-REAL-UID search parameter.
Timo Sirainen <tss@iki.fi>
parents: 17817
diff changeset
407 struct mail *real_mail;
98b5ccc539b5 lib-storage: Added X-REAL-UID search parameter.
Timo Sirainen <tss@iki.fi>
parents: 17817
diff changeset
408
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
409 if (mail_get_backend_mail(ctx->cur_mail, &real_mail) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
410 search_cur_mail_failed(ctx);
17949
98b5ccc539b5 lib-storage: Added X-REAL-UID search parameter.
Timo Sirainen <tss@iki.fi>
parents: 17817
diff changeset
411 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
412 }
17949
98b5ccc539b5 lib-storage: Added X-REAL-UID search parameter.
Timo Sirainen <tss@iki.fi>
parents: 17817
diff changeset
413 return seq_range_exists(&arg->value.seqset, real_mail->uid);
98b5ccc539b5 lib-storage: Added X-REAL-UID search parameter.
Timo Sirainen <tss@iki.fi>
parents: 17817
diff changeset
414 }
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
415 default:
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
416 return -1;
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
417 }
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
418 }
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
419
4907
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
420 static void search_cached_arg(struct mail_search_arg *arg,
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
421 struct index_search_context *ctx)
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
422 {
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
423 switch (search_arg_match_cached(ctx, arg)) {
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
424 case -1:
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
425 /* unknown */
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
426 break;
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
427 case 0:
1069
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
428 ARG_SET_RESULT(arg, 0);
525
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
429 break;
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
430 default:
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
431 ARG_SET_RESULT(arg, 1);
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
432 break;
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
433 }
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
434 }
2cb2e0a3423b Moved several fields from .imap.index file to .imap.index.data file. Fixed
Timo Sirainen <tss@iki.fi>
parents: 505
diff changeset
435
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
436 static int search_sent(enum mail_search_arg_type type, time_t search_time,
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
437 const unsigned char *sent_value, size_t sent_value_len)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
438 {
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
439 time_t sent_time;
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
440 int timezone_offset;
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
441
880
d5168cca2052 SEARCHing headers with "" value should always match if the header is found.
Timo Sirainen <tss@iki.fi>
parents: 825
diff changeset
442 if (sent_value == NULL)
d5168cca2052 SEARCHing headers with "" value should always match if the header is found.
Timo Sirainen <tss@iki.fi>
parents: 825
diff changeset
443 return 0;
d5168cca2052 SEARCHing headers with "" value should always match if the header is found.
Timo Sirainen <tss@iki.fi>
parents: 825
diff changeset
444
1340
087a9f4304ca s/latest IMAP4rev1 draft/RFC-3501/
Timo Sirainen <tss@iki.fi>
parents: 1322
diff changeset
445 /* NOTE: RFC-3501 specifies that timezone is ignored
935
bd2d99a3d601 Even more rfc822 date timezone fixes
Timo Sirainen <tss@iki.fi>
parents: 934
diff changeset
446 in searches. sent_time is returned as UTC, so change it. */
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
447 if (!message_date_parse(sent_value, sent_value_len,
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
448 &sent_time, &timezone_offset))
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
449 return 0;
1604
6a18854e6856 Still more timezone fixes
Timo Sirainen <tss@iki.fi>
parents: 1552
diff changeset
450 sent_time += timezone_offset * 60;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
451
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
452 switch (type) {
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
453 case SEARCH_BEFORE:
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
454 return sent_time < search_time;
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
455 case SEARCH_ON:
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
456 return sent_time >= search_time &&
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
457 sent_time < search_time + 3600*24;
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
458 case SEARCH_SINCE:
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
459 return sent_time >= search_time;
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
460 default:
546
e1254b838e0b Added --enable-asserts (default) and fixed some warnings when building
Timo Sirainen <tss@iki.fi>
parents: 534
diff changeset
461 i_unreached();
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
462 }
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
463 }
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
464
5525
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
465 static struct message_search_context *
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
466 msg_search_arg_context(struct index_search_context *ctx,
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
467 struct mail_search_arg *arg)
5504
9414946e1eb0 Message body search API changed to init/search/deinit. Searching now builds
Timo Sirainen <tss@iki.fi>
parents: 4949
diff changeset
468 {
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
469 enum message_search_flags flags = 0;
13927
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
470
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
471 if (arg->context == NULL) T_BEGIN {
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
472 string_t *dtc = t_str_new(128);
5504
9414946e1eb0 Message body search API changed to init/search/deinit. Searching now builds
Timo Sirainen <tss@iki.fi>
parents: 4949
diff changeset
473
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
474 if (ctx->mail_ctx.normalizer(arg->value.str,
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
475 strlen(arg->value.str), dtc) < 0)
13927
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
476 i_panic("search key not utf8: %s", arg->value.str);
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
477
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
478 if (arg->type == SEARCH_BODY)
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
479 flags |= MESSAGE_SEARCH_FLAG_SKIP_HEADERS;
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
480 /* we don't get here if arg is "", but dtc can be "" if it
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
481 only contains characters that we need to ignore. handle
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
482 those searches by returning them as non-matched. */
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
483 if (str_len(dtc) > 0) {
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
484 arg->context =
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
485 message_search_init(str_c(dtc),
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
486 ctx->mail_ctx.normalizer,
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
487 flags);
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
488 }
13927
bc3b343b1999 Don't assert-crash on mail search if decomposed titlecase of search key is empty.
Timo Sirainen <tss@iki.fi>
parents: 13832
diff changeset
489 } T_END;
12871
d399d5c65aca Changed header/body searching to do search key charset translation earlier.
Timo Sirainen <tss@iki.fi>
parents: 12870
diff changeset
490 return arg->context;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
491 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
492
7284
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
493 static void compress_lwsp(string_t *dest, const unsigned char *src,
21322
5ab8dc1a4a6f global: Change string position/length from unsigned int to size_t
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21066
diff changeset
494 size_t src_len)
7284
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
495 {
21322
5ab8dc1a4a6f global: Change string position/length from unsigned int to size_t
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21066
diff changeset
496 size_t i;
7284
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
497 bool prev_lwsp = TRUE;
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
498
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
499 for (i = 0; i < src_len; i++) {
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
500 if (IS_LWSP(src[i])) {
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
501 if (!prev_lwsp) {
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
502 prev_lwsp = TRUE;
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
503 str_append_c(dest, ' ');
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
504 }
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
505 } else {
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
506 prev_lwsp = FALSE;
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
507 str_append_c(dest, src[i]);
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
508 }
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
509 }
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
510 }
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
511
4907
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
512 static void search_header_arg(struct mail_search_arg *arg,
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
513 struct search_header_context *ctx)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
514 {
5525
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
515 struct message_search_context *msg_search_ctx;
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
516 struct message_block block;
6574
ab3a1a8ade55 Header searches didn't decode MIME encoded words.
Timo Sirainen <tss@iki.fi>
parents: 6492
diff changeset
517 struct message_header_line hdr;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
518 int ret;
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
519
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
520 /* first check that the field name matches to argument. */
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
521 switch (arg->type) {
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
522 case SEARCH_BEFORE:
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
523 case SEARCH_ON:
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
524 case SEARCH_SINCE:
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
525 if (arg->value.date_type != MAIL_SEARCH_DATE_TYPE_SENT)
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
526 return;
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
527
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
528 /* date is handled differently than others */
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
529 if (strcasecmp(ctx->hdr->name, "Date") == 0) {
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
530 if (ctx->hdr->continues) {
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
531 ctx->hdr->use_full_value = TRUE;
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
532 return;
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
533 }
5599
6789ba80419b Parse search dates already in IMAP code.
Timo Sirainen <tss@iki.fi>
parents: 5527
diff changeset
534 ret = search_sent(arg->type, arg->value.time,
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
535 ctx->hdr->full_value,
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
536 ctx->hdr->full_value_len);
1070
5c34c363a062 Fixes to SEARCH SENT*
Timo Sirainen <tss@iki.fi>
parents: 1069
diff changeset
537 ARG_SET_RESULT(arg, ret);
408
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
538 }
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
539 return;
e057845d94ca Dropped sent_time and alignment from MailIndexRecord. SEARCH can now use
Timo Sirainen <tss@iki.fi>
parents: 369
diff changeset
540
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
541 case SEARCH_HEADER:
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
542 case SEARCH_HEADER_ADDRESS:
7284
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
543 case SEARCH_HEADER_COMPRESS_LWSP:
10
82b7de533f98 s/user_data/context/ and some s/Data/Context/
Timo Sirainen <tss@iki.fi>
parents: 5
diff changeset
544 ctx->custom_header = TRUE;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
545
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
546 if (strcasecmp(ctx->hdr->name, arg->hdr_field_name) != 0)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
547 return;
5513
Timo Sirainen <tss@iki.fi>
parents: 5507
diff changeset
548 break;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
549 default:
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
550 return;
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
551 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
552
880
d5168cca2052 SEARCHing headers with "" value should always match if the header is found.
Timo Sirainen <tss@iki.fi>
parents: 825
diff changeset
553 if (arg->value.str[0] == '\0') {
d5168cca2052 SEARCHing headers with "" value should always match if the header is found.
Timo Sirainen <tss@iki.fi>
parents: 825
diff changeset
554 /* we're just testing existence of the field. always matches. */
5525
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
555 ARG_SET_RESULT(arg, 1);
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
556 return;
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
557 }
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
558
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
559 if (ctx->hdr->continues) {
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
560 ctx->hdr->use_full_value = TRUE;
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
561 return;
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
562 }
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
563
21389
59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(*p)) with the new i_zero() macro.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 21322
diff changeset
564 i_zero(&block);
7281
e5b840b3c5c2 Header searches shouldn't include the header name or ":" in matching.
Timo Sirainen <tss@iki.fi>
parents: 7279
diff changeset
565
e5b840b3c5c2 Header searches shouldn't include the header name or ":" in matching.
Timo Sirainen <tss@iki.fi>
parents: 7279
diff changeset
566 /* We're searching only for values, so drop header name and middle
e5b840b3c5c2 Header searches shouldn't include the header name or ":" in matching.
Timo Sirainen <tss@iki.fi>
parents: 7279
diff changeset
567 parts. We use header searching so that MIME words will be decoded. */
e5b840b3c5c2 Header searches shouldn't include the header name or ":" in matching.
Timo Sirainen <tss@iki.fi>
parents: 7279
diff changeset
568 hdr = *ctx->hdr;
e5b840b3c5c2 Header searches shouldn't include the header name or ":" in matching.
Timo Sirainen <tss@iki.fi>
parents: 7279
diff changeset
569 hdr.name = ""; hdr.name_len = 0;
e5b840b3c5c2 Header searches shouldn't include the header name or ":" in matching.
Timo Sirainen <tss@iki.fi>
parents: 7279
diff changeset
570 hdr.middle_len = 0;
e5b840b3c5c2 Header searches shouldn't include the header name or ":" in matching.
Timo Sirainen <tss@iki.fi>
parents: 7279
diff changeset
571 block.hdr = &hdr;
e5b840b3c5c2 Header searches shouldn't include the header name or ":" in matching.
Timo Sirainen <tss@iki.fi>
parents: 7279
diff changeset
572
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
573 msg_search_ctx = msg_search_arg_context(ctx->index_ctx, arg);
5525
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
574 if (msg_search_ctx == NULL)
7284
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
575 return;
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
576
19757
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
577 if (!ctx->decoded_block_set) { T_BEGIN {
7284
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
578 struct message_address *addr;
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
579 string_t *str;
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
580
7284
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
581 switch (arg->type) {
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
582 case SEARCH_HEADER:
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
583 /* simple match */
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
584 break;
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
585 case SEARCH_HEADER_ADDRESS:
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
586 /* we have to match against normalized address */
6940
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents: 6899
diff changeset
587 addr = message_address_parse(pool_datastack_create(),
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents: 6899
diff changeset
588 ctx->hdr->full_value,
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents: 6899
diff changeset
589 ctx->hdr->full_value_len,
15904
d3cf06639864 Replaced all -1U and (unsigned int)-1 with UINT_MAX.
Timo Sirainen <tss@iki.fi>
parents: 15715
diff changeset
590 UINT_MAX, TRUE);
6940
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents: 6899
diff changeset
591 str = t_str_new(ctx->hdr->value_len);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents: 6899
diff changeset
592 message_address_write(str, addr);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents: 6899
diff changeset
593 hdr.value = hdr.full_value = str_data(str);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME*() macros.
Timo Sirainen <tss@iki.fi>
parents: 6899
diff changeset
594 hdr.value_len = hdr.full_value_len = str_len(str);
7284
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
595 break;
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
596 case SEARCH_HEADER_COMPRESS_LWSP:
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
597 /* convert LWSP to single spaces */
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
598 str = t_str_new(hdr.full_value_len);
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
599 compress_lwsp(str, hdr.full_value, hdr.full_value_len);
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
600 hdr.value = hdr.full_value = str_data(str);
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
601 hdr.value_len = hdr.full_value_len = str_len(str);
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
602 break;
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
603 default:
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
604 i_unreached();
346d4a44c2f4 SEARCH SUBJECT now changes LWSP to spaces to match ENVELOPE's subject field.
Timo Sirainen <tss@iki.fi>
parents: 7281
diff changeset
605 }
19757
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
606 ret = message_search_more_get_decoded(msg_search_ctx, &block,
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
607 &ctx->decoded_block) ? 1 : 0;
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
608 ctx->decoded_block_set = TRUE;
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
609 } T_END; } else {
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
610 /* this block was already decoded and saved by an earlier
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
611 search arg. use the already-decoded block to avoid
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
612 duplicating work. */
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
613 ret = message_search_more_decoded(msg_search_ctx,
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
614 &ctx->decoded_block) ? 1 : 0;
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
615 }
568
f2aa58c2afd0 SEARCH CHARSET support. Currently we do it through iconv() and only ASCII
Timo Sirainen <tss@iki.fi>
parents: 559
diff changeset
616
11558
3e50223d0014 lib-storage: Fixed header searches to work correctly when there are multiple headers with same name.
Timo Sirainen <tss@iki.fi>
parents: 11291
diff changeset
617 /* there may be multiple headers. don't mark this failed yet. */
3e50223d0014 lib-storage: Fixed header searches to work correctly when there are multiple headers with same name.
Timo Sirainen <tss@iki.fi>
parents: 11291
diff changeset
618 if (ret > 0)
3e50223d0014 lib-storage: Fixed header searches to work correctly when there are multiple headers with same name.
Timo Sirainen <tss@iki.fi>
parents: 11291
diff changeset
619 ARG_SET_RESULT(arg, 1);
1069
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
620 }
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
621
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
622 static void search_header_unmatch(struct mail_search_arg *arg,
14921
d3db2ba15d00 Removed CONTEXT_TYPE_SAFETY macro and reimplemented its functionality better.
Timo Sirainen <tss@iki.fi>
parents: 14738
diff changeset
623 struct search_header_context *ctx ATTR_UNUSED)
1069
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
624 {
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
625 switch (arg->type) {
11141
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
626 case SEARCH_BEFORE:
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
627 case SEARCH_ON:
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
628 case SEARCH_SINCE:
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
629 if (arg->value.date_type != MAIL_SEARCH_DATE_TYPE_SENT)
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
630 break;
4ade4d8c66c5 lib-storage: Added support for searching save date.
Timo Sirainen <tss@iki.fi>
parents: 11000
diff changeset
631
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
632 if (arg->match_not) {
1070
5c34c363a062 Fixes to SEARCH SENT*
Timo Sirainen <tss@iki.fi>
parents: 1069
diff changeset
633 /* date header not found, so we match only for
5c34c363a062 Fixes to SEARCH SENT*
Timo Sirainen <tss@iki.fi>
parents: 1069
diff changeset
634 NOT searches */
5c34c363a062 Fixes to SEARCH SENT*
Timo Sirainen <tss@iki.fi>
parents: 1069
diff changeset
635 ARG_SET_RESULT(arg, 0);
5c34c363a062 Fixes to SEARCH SENT*
Timo Sirainen <tss@iki.fi>
parents: 1069
diff changeset
636 }
5c34c363a062 Fixes to SEARCH SENT*
Timo Sirainen <tss@iki.fi>
parents: 1069
diff changeset
637 break;
1069
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
638 case SEARCH_HEADER:
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
639 case SEARCH_HEADER_ADDRESS:
7328
e0e212e7deeb SEARCH NOT SUBJECT didn't work correctly.
Timo Sirainen <tss@iki.fi>
parents: 7284
diff changeset
640 case SEARCH_HEADER_COMPRESS_LWSP:
1069
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
641 ARG_SET_RESULT(arg, 0);
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
642 break;
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
643 default:
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
644 break;
8024a2cdc406 Several search fixes, mostly related to matching multiple conditions.
Timo Sirainen <tss@iki.fi>
parents: 1066
diff changeset
645 }
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
646 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
647
4907
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
648 static void search_header(struct message_header_line *hdr,
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
649 struct search_header_context *ctx)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
650 {
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
651 if (hdr == NULL) {
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
652 /* end of headers, mark all unknown SEARCH_HEADERs unmatched */
14682
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14676
diff changeset
653 (void)mail_search_args_foreach(ctx->args, search_header_unmatch,
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14676
diff changeset
654 ctx);
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
655 return;
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
656 }
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
657
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
658 if (hdr->eoh)
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
659 return;
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
660
3325
38eaa792c405 Don't call index_mail_parse_header() twice when searching headers. Caused
Timo Sirainen <tss@iki.fi>
parents: 3301
diff changeset
661 if (ctx->parse_headers)
12871
d399d5c65aca Changed header/body searching to do search key charset translation earlier.
Timo Sirainen <tss@iki.fi>
parents: 12870
diff changeset
662 index_mail_parse_header(NULL, hdr, ctx->imail);
924
4f697dde0fca THREAD=REFERENCES implementation. Doesn't crash, but I'm not sure how
Timo Sirainen <tss@iki.fi>
parents: 903
diff changeset
663
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
664 if (ctx->custom_header || strcasecmp(hdr->name, "Date") == 0) {
1322
97f8c00b8d4c Better handling for multiline headers. Before we skipped headers larger than
Timo Sirainen <tss@iki.fi>
parents: 1070
diff changeset
665 ctx->hdr = hdr;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
666
19757
d7e18400a7ce lib-storage: Avoid duplicate work when a search query has multiple header/body keys.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19552
diff changeset
667 ctx->decoded_block_set = FALSE;
10
82b7de533f98 s/user_data/context/ and some s/Data/Context/
Timo Sirainen <tss@iki.fi>
parents: 5
diff changeset
668 ctx->custom_header = FALSE;
14682
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14676
diff changeset
669 (void)mail_search_args_foreach(ctx->args, search_header_arg, ctx);
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
670 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
671 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
672
4907
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
673 static void search_body(struct mail_search_arg *arg,
5b4c9b20eba0 Replaced void *context from a lot of callbacks with the actual context
Timo Sirainen <tss@iki.fi>
parents: 4635
diff changeset
674 struct search_body_context *ctx)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
675 {
5525
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
676 struct message_search_context *msg_search_ctx;
19887
c5c275bfc07c lib-mail: message_search_msg() passes through message_parser_deinit_from_parts()'s error string
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19886
diff changeset
677 const char *error;
3863
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool.
Timo Sirainen <tss@iki.fi>
parents: 3795
diff changeset
678 int ret;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
679
4942
b382b21409b5 Added support for X-BODY-FAST and X-TEXT-FAST SEARCH arguments. FTS plugin
Timo Sirainen <tss@iki.fi>
parents: 4939
diff changeset
680 switch (arg->type) {
b382b21409b5 Added support for X-BODY-FAST and X-TEXT-FAST SEARCH arguments. FTS plugin
Timo Sirainen <tss@iki.fi>
parents: 4939
diff changeset
681 case SEARCH_BODY:
b382b21409b5 Added support for X-BODY-FAST and X-TEXT-FAST SEARCH arguments. FTS plugin
Timo Sirainen <tss@iki.fi>
parents: 4939
diff changeset
682 case SEARCH_TEXT:
b382b21409b5 Added support for X-BODY-FAST and X-TEXT-FAST SEARCH arguments. FTS plugin
Timo Sirainen <tss@iki.fi>
parents: 4939
diff changeset
683 break;
b382b21409b5 Added support for X-BODY-FAST and X-TEXT-FAST SEARCH arguments. FTS plugin
Timo Sirainen <tss@iki.fi>
parents: 4939
diff changeset
684 default:
3795
e6c42de08336 Allow message_body_search() to return "message_part is broken" error. If it
Timo Sirainen <tss@iki.fi>
parents: 3467
diff changeset
685 return;
4942
b382b21409b5 Added support for X-BODY-FAST and X-TEXT-FAST SEARCH arguments. FTS plugin
Timo Sirainen <tss@iki.fi>
parents: 4939
diff changeset
686 }
3795
e6c42de08336 Allow message_body_search() to return "message_part is broken" error. If it
Timo Sirainen <tss@iki.fi>
parents: 3467
diff changeset
687
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
688 msg_search_ctx = msg_search_arg_context(ctx->index_ctx, arg);
5525
94bc3fbdecd5 Removed message body/header searchers. They're now combined into one
Timo Sirainen <tss@iki.fi>
parents: 5522
diff changeset
689 if (msg_search_ctx == NULL) {
5504
9414946e1eb0 Message body search API changed to init/search/deinit. Searching now builds
Timo Sirainen <tss@iki.fi>
parents: 4949
diff changeset
690 ARG_SET_RESULT(arg, 0);
9414946e1eb0 Message body search API changed to init/search/deinit. Searching now builds
Timo Sirainen <tss@iki.fi>
parents: 4949
diff changeset
691 return;
9414946e1eb0 Message body search API changed to init/search/deinit. Searching now builds
Timo Sirainen <tss@iki.fi>
parents: 4949
diff changeset
692 }
9414946e1eb0 Message body search API changed to init/search/deinit. Searching now builds
Timo Sirainen <tss@iki.fi>
parents: 4949
diff changeset
693
3795
e6c42de08336 Allow message_body_search() to return "message_part is broken" error. If it
Timo Sirainen <tss@iki.fi>
parents: 3467
diff changeset
694 i_stream_seek(ctx->input, 0);
19887
c5c275bfc07c lib-mail: message_search_msg() passes through message_parser_deinit_from_parts()'s error string
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19886
diff changeset
695 ret = message_search_msg(msg_search_ctx, ctx->input, ctx->part, &error);
7241
a6c066f50877 Added mail_set_cache_corrupted() to rebuild the cache file and force
Timo Sirainen <tss@iki.fi>
parents: 7226
diff changeset
696 if (ret < 0 && ctx->input->stream_errno == 0) {
a6c066f50877 Added mail_set_cache_corrupted() to rebuild the cache file and force
Timo Sirainen <tss@iki.fi>
parents: 7226
diff changeset
697 /* try again without cached parts */
19889
0f5c55eaa401 lib-storage: When cached mime.parts is detected to be inconsistent, log it as hex-encoded.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19887
diff changeset
698 index_mail_set_message_parts_corrupted(ctx->index_ctx->cur_mail, error);
3795
e6c42de08336 Allow message_body_search() to return "message_part is broken" error. If it
Timo Sirainen <tss@iki.fi>
parents: 3467
diff changeset
699
5504
9414946e1eb0 Message body search API changed to init/search/deinit. Searching now builds
Timo Sirainen <tss@iki.fi>
parents: 4949
diff changeset
700 i_stream_seek(ctx->input, 0);
19887
c5c275bfc07c lib-mail: message_search_msg() passes through message_parser_deinit_from_parts()'s error string
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 19886
diff changeset
701 ret = message_search_msg(msg_search_ctx, ctx->input, NULL, &error);
7241
a6c066f50877 Added mail_set_cache_corrupted() to rebuild the cache file and force
Timo Sirainen <tss@iki.fi>
parents: 7226
diff changeset
702 i_assert(ret >= 0 || ctx->input->stream_errno != 0);
3795
e6c42de08336 Allow message_body_search() to return "message_part is broken" error. If it
Timo Sirainen <tss@iki.fi>
parents: 3467
diff changeset
703 }
15266
99965e11d9e1 lib-storage: Check and log stream errors when parsing/searching messages.
Timo Sirainen <tss@iki.fi>
parents: 15079
diff changeset
704 if (ctx->input->stream_errno != 0) {
99965e11d9e1 lib-storage: Check and log stream errors when parsing/searching messages.
Timo Sirainen <tss@iki.fi>
parents: 15079
diff changeset
705 mail_storage_set_critical(ctx->index_ctx->box->storage,
17996
7681fcd1ba43 Use i_stream_get_error() wherever possible instead of %m
Timo Sirainen <tss@iki.fi>
parents: 17949
diff changeset
706 "read(%s) failed: %s", i_stream_get_name(ctx->input),
7681fcd1ba43 Use i_stream_get_error() wherever possible instead of %m
Timo Sirainen <tss@iki.fi>
parents: 17949
diff changeset
707 i_stream_get_error(ctx->input));
15266
99965e11d9e1 lib-storage: Check and log stream errors when parsing/searching messages.
Timo Sirainen <tss@iki.fi>
parents: 15079
diff changeset
708 }
290
3dcc2275b4ca IOBuffer cleanup, hopefully fixes some mbox problems.
Timo Sirainen <tss@iki.fi>
parents: 223
diff changeset
709
12254
41454ed5a3eb search: If body search fails due to I/O error, keep the result as "unknown".
Timo Sirainen <tss@iki.fi>
parents: 11887
diff changeset
710 ARG_SET_RESULT(arg, ret);
5
1b34ec11fff8 Message data is parsed in blocks (no longer entirely mmap()ed). Several
Timo Sirainen <tss@iki.fi>
parents: 0
diff changeset
711 }
1b34ec11fff8 Message data is parsed in blocks (no longer entirely mmap()ed). Several
Timo Sirainen <tss@iki.fi>
parents: 0
diff changeset
712
9162
3727bfc568b9 Search: Perform all lookups from cache before attempting any uncached lookups.
Timo Sirainen <tss@iki.fi>
parents: 8590
diff changeset
713 static int search_arg_match_text(struct mail_search_arg *args,
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
714 struct index_search_context *ctx)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
715 {
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
716 const enum message_header_parser_flags hdr_parser_flags =
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
717 MESSAGE_HEADER_PARSER_FLAG_CLEAN_ONELINE;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
718 struct index_mail *imail = (struct index_mail *)ctx->cur_mail;
17212
e47b3e215dec lib-storage: Replaced mail_get_real_mail() with mail_get_backend_mail() that can fail.
Timo Sirainen <tss@iki.fi>
parents: 17130
diff changeset
719 struct mail *real_mail;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
720 struct istream *input = NULL;
2327
7d02e2a7672d Header caching redesigned. New design allows caching decisions per field, so
Timo Sirainen <tss@iki.fi>
parents: 2298
diff changeset
721 struct mailbox_header_lookup_ctx *headers_ctx;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
722 struct search_header_context hdr_ctx;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
723 struct search_body_context body_ctx;
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
724 const char *const *headers;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
725 bool have_headers, have_body, failed = FALSE;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
726 int ret;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
727
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
728 /* first check what we need to use */
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
729 headers = mail_search_args_analyze(args, &have_headers, &have_body);
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
730 if (!have_headers && !have_body)
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
731 return -1;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
732
21389
59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(*p)) with the new i_zero() macro.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 21322
diff changeset
733 i_zero(&hdr_ctx);
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
734 hdr_ctx.index_ctx = ctx;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
735 /* hdr_ctx.imail is different from imail for mails in
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
736 virtual mailboxes */
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
737 if (mail_get_backend_mail(ctx->cur_mail, &real_mail) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
738 search_cur_mail_failed(ctx);
17212
e47b3e215dec lib-storage: Replaced mail_get_real_mail() with mail_get_backend_mail() that can fail.
Timo Sirainen <tss@iki.fi>
parents: 17130
diff changeset
739 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
740 }
17212
e47b3e215dec lib-storage: Replaced mail_get_real_mail() with mail_get_backend_mail() that can fail.
Timo Sirainen <tss@iki.fi>
parents: 17130
diff changeset
741 hdr_ctx.imail = (struct index_mail *)real_mail;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
742 hdr_ctx.custom_header = TRUE;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
743 hdr_ctx.args = args;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
744
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
745 headers_ctx = headers == NULL ? NULL :
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
746 mailbox_header_lookup_init(ctx->box, headers);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
747 if (headers != NULL &&
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
748 (!have_body ||
16841
4b0a736bc40c lib-storage: If search finds a match to a cached header, don't open the body stream.
Timo Sirainen <tss@iki.fi>
parents: 16840
diff changeset
749 ctx->cur_mail->lookup_abort != MAIL_LOOKUP_ABORT_NEVER)) {
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
750 /* try to look up the specified headers from cache */
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
751 i_assert(*headers != NULL);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
752
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
753 if (mail_get_header_stream(ctx->cur_mail, headers_ctx,
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
754 &input) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
755 search_cur_mail_failed(ctx);
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
756 failed = TRUE;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
757 } else {
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
758 message_parse_header(input, NULL, hdr_parser_flags,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
759 search_header, &hdr_ctx);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
760 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
761 input = NULL;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
762 } else if (have_headers) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
763 /* we need to read the entire header */
16840
78081ecd7eb3 lib-storage: Header+body searches might not have searched the body in some cases.
Timo Sirainen <tss@iki.fi>
parents: 16389
diff changeset
764 ret = have_body ?
20560
3a71ed48cdf7 Use mail_get_*stream_because() wherever possible.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20082
diff changeset
765 mail_get_stream_because(ctx->cur_mail, NULL, NULL, "search", &input) :
3a71ed48cdf7 Use mail_get_*stream_because() wherever possible.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20082
diff changeset
766 mail_get_hdr_stream_because(ctx->cur_mail, NULL, "search", &input);
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
767 if (ret < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
768 search_cur_mail_failed(ctx);
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
769 failed = TRUE;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
770 } else {
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
771 hdr_ctx.parse_headers =
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
772 index_mail_want_parse_headers(hdr_ctx.imail);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
773 if (hdr_ctx.parse_headers) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
774 index_mail_parse_header_init(hdr_ctx.imail,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
775 headers_ctx);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
776 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
777 message_parse_header(input, NULL, hdr_parser_flags,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
778 search_header, &hdr_ctx);
15266
99965e11d9e1 lib-storage: Check and log stream errors when parsing/searching messages.
Timo Sirainen <tss@iki.fi>
parents: 15079
diff changeset
779 if (input->stream_errno != 0) {
99965e11d9e1 lib-storage: Check and log stream errors when parsing/searching messages.
Timo Sirainen <tss@iki.fi>
parents: 15079
diff changeset
780 mail_storage_set_critical(ctx->box->storage,
17996
7681fcd1ba43 Use i_stream_get_error() wherever possible instead of %m
Timo Sirainen <tss@iki.fi>
parents: 17949
diff changeset
781 "read(%s) failed: %s",
7681fcd1ba43 Use i_stream_get_error() wherever possible instead of %m
Timo Sirainen <tss@iki.fi>
parents: 17949
diff changeset
782 i_stream_get_name(input),
7681fcd1ba43 Use i_stream_get_error() wherever possible instead of %m
Timo Sirainen <tss@iki.fi>
parents: 17949
diff changeset
783 i_stream_get_error(input));
15266
99965e11d9e1 lib-storage: Check and log stream errors when parsing/searching messages.
Timo Sirainen <tss@iki.fi>
parents: 15079
diff changeset
784 failed = TRUE;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
785 search_set_failed(ctx);
15266
99965e11d9e1 lib-storage: Check and log stream errors when parsing/searching messages.
Timo Sirainen <tss@iki.fi>
parents: 15079
diff changeset
786 }
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
787 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
788 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
789 if (headers_ctx != NULL)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
790 mailbox_header_lookup_unref(&headers_ctx);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
791
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
792 if (failed) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
793 /* opening mail failed. maybe because of lookup_abort.
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
794 update access_parts for prefetching */
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
795 if (have_body)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
796 imail->data.access_part |= READ_HDR | READ_BODY;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
797 else
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
798 imail->data.access_part |= READ_HDR;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
799 return -1;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
800 }
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
801
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
802 if (have_headers) {
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
803 /* see if the header search succeeded in finishing the search */
14921
d3db2ba15d00 Removed CONTEXT_TYPE_SAFETY macro and reimplemented its functionality better.
Timo Sirainen <tss@iki.fi>
parents: 14738
diff changeset
804 ret = mail_search_args_foreach(args, search_none, (void *)NULL);
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
805 if (ret >= 0 || !have_body)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
806 return ret;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
807 }
5
1b34ec11fff8 Message data is parsed in blocks (no longer entirely mmap()ed). Several
Timo Sirainen <tss@iki.fi>
parents: 0
diff changeset
808
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
809 i_assert(have_body);
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
810
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
811 if (ctx->cur_mail->lookup_abort != MAIL_LOOKUP_ABORT_NEVER) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
812 imail->data.access_part |= READ_HDR | READ_BODY;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
813 return -1;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
814 }
715
3e952f64b2ec Non-envelope-cached header searching wasn't working. Also now search works
Timo Sirainen <tss@iki.fi>
parents: 678
diff changeset
815
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
816 if (input == NULL) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
817 /* we didn't search headers. */
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
818 struct message_size hdr_size;
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
819
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
820 if (mail_get_stream_because(ctx->cur_mail, &hdr_size, NULL, "search", &input) < 0) {
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
821 search_cur_mail_failed(ctx);
9162
3727bfc568b9 Search: Perform all lookups from cache before attempting any uncached lookups.
Timo Sirainen <tss@iki.fi>
parents: 8590
diff changeset
822 return -1;
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
823 }
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
824 i_stream_seek(input, hdr_size.physical_size);
5
1b34ec11fff8 Message data is parsed in blocks (no longer entirely mmap()ed). Several
Timo Sirainen <tss@iki.fi>
parents: 0
diff changeset
825 }
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
826
21389
59437f8764c6 global: Replaced all instances of memset(p, 0, sizeof(*p)) with the new i_zero() macro.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 21322
diff changeset
827 i_zero(&body_ctx);
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
828 body_ctx.index_ctx = ctx;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
829 body_ctx.input = input;
21600
236081fab146 lib-storage: Search optimization - avoid parsing message_parts unnecessarily
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21599
diff changeset
830 /* Get parts if they already exist in cache. If they don't,
236081fab146 lib-storage: Search optimization - avoid parsing message_parts unnecessarily
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21599
diff changeset
831 message-search will parse the mail automatically. */
236081fab146 lib-storage: Search optimization - avoid parsing message_parts unnecessarily
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21599
diff changeset
832 ctx->cur_mail->lookup_abort = MAIL_LOOKUP_ABORT_NOT_IN_CACHE;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
833 (void)mail_get_parts(ctx->cur_mail, &body_ctx.part);
21600
236081fab146 lib-storage: Search optimization - avoid parsing message_parts unnecessarily
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21599
diff changeset
834 ctx->cur_mail->lookup_abort = MAIL_LOOKUP_ABORT_NEVER;
608
debb8468514e SEARCH CHARSET now works properly with message bodies, and in general body
Timo Sirainen <tss@iki.fi>
parents: 569
diff changeset
835
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
836 return mail_search_args_foreach(args, search_body, &body_ctx);
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
837 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
838
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
839 static bool
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
840 search_msgset_fix_limits(unsigned int messages_count,
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
841 ARRAY_TYPE(seq_range) *seqset, bool match_not)
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
842 {
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
843 struct seq_range *range;
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
844 unsigned int count;
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
845
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
846 i_assert(messages_count > 0);
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
847
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
848 range = array_get_modifiable(seqset, &count);
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
849 if (count > 0) {
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
850 i_assert(range[0].seq1 != 0);
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
851 if (range[count-1].seq2 == (uint32_t)-1) {
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
852 /* "*" used, make sure the last message is in the range
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
853 (e.g. with count+1:* we still want to include it) */
14676
69ba6977bed8 seq_range_array_add() API changed. Added other functions to provide the less common use cases.
Timo Sirainen <tss@iki.fi>
parents: 14629
diff changeset
854 seq_range_array_add(seqset, messages_count);
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
855 }
11887
05031751cc21 Fixing my english: s/non-?existing/nonexistent/
Timo Sirainen <tss@iki.fi>
parents: 11558
diff changeset
856 /* remove all nonexistent messages */
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
857 seq_range_array_remove_range(seqset, messages_count + 1,
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
858 (uint32_t)-1);
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
859 }
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
860 if (!match_not)
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
861 return array_count(seqset) > 0;
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
862 else {
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
863 /* if all messages are in the range, it can't match */
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
864 range = array_get_modifiable(seqset, &count);
13146
0e64f0217feb lib-storage: Fixed crashing on "NOT <nonexistent sequence>" search.
Timo Sirainen <tss@iki.fi>
parents: 12822
diff changeset
865 return count == 0 || range[0].seq1 != 1 ||
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
866 range[count-1].seq2 != messages_count;
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
867 }
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
868 }
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
869
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
870 static void
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
871 search_msgset_fix(unsigned int messages_count,
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
872 ARRAY_TYPE(seq_range) *seqset,
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
873 uint32_t *seq1_r, uint32_t *seq2_r, bool match_not)
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
874 {
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
875 const struct seq_range *range;
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
876 unsigned int count;
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
877 uint32_t min_seq, max_seq;
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
878
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
879 if (!search_msgset_fix_limits(messages_count, seqset, match_not)) {
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
880 *seq1_r = (uint32_t)-1;
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
881 *seq2_r = 0;
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
882 return;
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
883 }
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
884
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
885 range = array_get(seqset, &count);
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
886 if (!match_not) {
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
887 min_seq = range[0].seq1;
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
888 max_seq = range[count-1].seq2;
13146
0e64f0217feb lib-storage: Fixed crashing on "NOT <nonexistent sequence>" search.
Timo Sirainen <tss@iki.fi>
parents: 12822
diff changeset
889 } else if (count == 0) {
0e64f0217feb lib-storage: Fixed crashing on "NOT <nonexistent sequence>" search.
Timo Sirainen <tss@iki.fi>
parents: 12822
diff changeset
890 /* matches all messages */
0e64f0217feb lib-storage: Fixed crashing on "NOT <nonexistent sequence>" search.
Timo Sirainen <tss@iki.fi>
parents: 12822
diff changeset
891 min_seq = 1;
0e64f0217feb lib-storage: Fixed crashing on "NOT <nonexistent sequence>" search.
Timo Sirainen <tss@iki.fi>
parents: 12822
diff changeset
892 max_seq = messages_count;
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
893 } else {
7619
56f55bd35aa5 Moved IMAP messageset handling to lib-imap/ and searching to lib-storage/.
Timo Sirainen <tss@iki.fi>
parents: 7618
diff changeset
894 min_seq = range[0].seq1 > 1 ? 1 : range[0].seq2 + 1;
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
895 max_seq = range[count-1].seq2 < messages_count ?
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
896 messages_count : range[count-1].seq1 - 1;
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
897 if (min_seq > max_seq) {
5605
8b97d6bbff35 Don't try to handle 0 in messagesets by returning a nice error message.
Timo Sirainen <tss@iki.fi>
parents: 5599
diff changeset
898 *seq1_r = (uint32_t)-1;
8b97d6bbff35 Don't try to handle 0 in messagesets by returning a nice error message.
Timo Sirainen <tss@iki.fi>
parents: 5599
diff changeset
899 *seq2_r = 0;
8b97d6bbff35 Don't try to handle 0 in messagesets by returning a nice error message.
Timo Sirainen <tss@iki.fi>
parents: 5599
diff changeset
900 return;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
901 }
962
3b139ccc1858 SEARCH didn't properly complain about invalid messagesets. high:low never
Timo Sirainen <tss@iki.fi>
parents: 951
diff changeset
902 }
4439
12b5fbb3a05c Searching (storing, fetching) with sequence sets containing commas was
Timo Sirainen <tss@iki.fi>
parents: 4438
diff changeset
903
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
904 if (*seq1_r < min_seq || *seq1_r == 0)
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
905 *seq1_r = min_seq;
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
906 if (*seq2_r > max_seq)
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
907 *seq2_r = max_seq;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
908 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
909
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
910 static void search_or_parse_msgset_args(unsigned int messages_count,
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
911 struct mail_search_arg *args,
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
912 uint32_t *seq1_r, uint32_t *seq2_r)
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
913 {
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
914 uint32_t seq1, seq2, min_seq1 = 0, max_seq2 = 0;
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
915
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
916 for (; args != NULL; args = args->next) {
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
917 seq1 = 1; seq2 = messages_count;
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
918
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
919 switch (args->type) {
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
920 case SEARCH_SUB:
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
921 i_assert(!args->match_not);
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
922 search_parse_msgset_args(messages_count,
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
923 args->value.subargs,
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
924 &seq1, &seq2);
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
925 break;
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
926 case SEARCH_OR:
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
927 i_assert(!args->match_not);
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
928 search_or_parse_msgset_args(messages_count,
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
929 args->value.subargs,
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
930 &seq1, &seq2);
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
931 break;
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
932 case SEARCH_SEQSET:
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
933 search_msgset_fix(messages_count, &args->value.seqset,
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
934 &seq1, &seq2, args->match_not);
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
935 break;
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
936 default:
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
937 break;
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
938 }
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
939
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
940 if (min_seq1 == 0) {
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
941 min_seq1 = seq1;
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
942 max_seq2 = seq2;
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
943 } else {
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
944 if (seq1 < min_seq1)
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
945 min_seq1 = seq1;
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
946 if (seq2 > max_seq2)
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
947 max_seq2 = seq2;
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
948 }
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
949 }
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
950 i_assert(min_seq1 != 0);
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
951
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
952 if (min_seq1 > *seq1_r)
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
953 *seq1_r = min_seq1;
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
954 if (max_seq2 < *seq2_r)
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
955 *seq2_r = max_seq2;
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
956 }
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
957
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
958 static void search_parse_msgset_args(unsigned int messages_count,
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
959 struct mail_search_arg *args,
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
960 uint32_t *seq1_r, uint32_t *seq2_r)
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
961 {
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
962 for (; args != NULL; args = args->next) {
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
963 switch (args->type) {
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
964 case SEARCH_SUB:
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
965 i_assert(!args->match_not);
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
966 search_parse_msgset_args(messages_count,
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
967 args->value.subargs,
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
968 seq1_r, seq2_r);
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
969 break;
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
970 case SEARCH_OR:
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
971 /* go through our children and use the widest seqset
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
972 range */
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
973 i_assert(!args->match_not);
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
974 search_or_parse_msgset_args(messages_count,
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
975 args->value.subargs,
6485
2eff72b212fe Simplify search tree by canonicalizing message sets, converting NOT away
Timo Sirainen <tss@iki.fi>
parents: 6457
diff changeset
976 seq1_r, seq2_r);
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
977 break;
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
978 case SEARCH_SEQSET:
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
979 search_msgset_fix(messages_count, &args->value.seqset,
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
980 seq1_r, seq2_r, args->match_not);
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
981 break;
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
982 default:
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
983 break;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
984 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
985 }
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
986 }
962
3b139ccc1858 SEARCH didn't properly complain about invalid messagesets. high:low never
Timo Sirainen <tss@iki.fi>
parents: 951
diff changeset
987
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
988 static void search_limit_lowwater(struct index_search_context *ctx,
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
989 uint32_t uid_lowwater, uint32_t *first_seq)
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
990 {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
991 uint32_t seq1, seq2;
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
992
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
993 if (uid_lowwater == 0)
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
994 return;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
995
14682
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14676
diff changeset
996 (void)mail_index_lookup_seq_range(ctx->view, uid_lowwater, (uint32_t)-1,
d0d7b810646b Make sure we check all the functions' return values. Minor API changes to simplify this.
Timo Sirainen <tss@iki.fi>
parents: 14676
diff changeset
997 &seq1, &seq2);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
998 if (*first_seq < seq1)
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
999 *first_seq = seq1;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1000 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1001
16389
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1002 static bool search_limit_by_hdr(struct index_search_context *ctx,
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1003 struct mail_search_arg *args,
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1004 uint32_t *seq1, uint32_t *seq2)
442
2a25bf21557b SEARCH SEEN, UNSEEN, DELETED, UNDELETED, RECENT, UNRECENT optimizations.
Timo Sirainen <tss@iki.fi>
parents: 441
diff changeset
1005 {
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1006 const struct mail_index_header *hdr;
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
1007 enum mail_flags pvt_flags_mask;
16389
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1008 uint64_t highest_modseq;
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
1009
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1010 hdr = mail_index_get_header(ctx->view);
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1011 /* we can't trust that private view's header is fully up to date,
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1012 so do this optimization only for non-private flags */
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1013 pvt_flags_mask = ctx->box->view_pvt == NULL ? 0 :
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1014 mailbox_get_private_flags_mask(ctx->box);
14704
04c89bb18bdd lib-storage: Fixed searching message flags with private flags index.
Timo Sirainen <tss@iki.fi>
parents: 14686
diff changeset
1015
442
2a25bf21557b SEARCH SEEN, UNSEEN, DELETED, UNDELETED, RECENT, UNRECENT optimizations.
Timo Sirainen <tss@iki.fi>
parents: 441
diff changeset
1016 for (; args != NULL; args = args->next) {
16389
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1017 switch (args->type) {
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1018 case SEARCH_ALL:
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1019 if (args->match_not) {
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1020 /* NOT ALL - pointless noop query */
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1021 return FALSE;
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
1022 }
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
1023 continue;
16389
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1024 case SEARCH_MODSEQ:
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1025 /* MODSEQ higher than current HIGHESTMODSEQ? */
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1026 highest_modseq = mail_index_modseq_get_highest(ctx->view);
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1027 if (args->value.modseq->modseq > highest_modseq)
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1028 return FALSE;
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1029 continue;
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1030 default:
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1031 continue;
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1032 case SEARCH_FLAGS:
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1033 break;
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
1034 }
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1035 if ((args->value.flags & MAIL_SEEN) != 0 &&
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1036 (pvt_flags_mask & MAIL_SEEN) == 0) {
442
2a25bf21557b SEARCH SEEN, UNSEEN, DELETED, UNDELETED, RECENT, UNRECENT optimizations.
Timo Sirainen <tss@iki.fi>
parents: 441
diff changeset
1037 /* SEEN with 0 seen? */
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1038 if (!args->match_not && hdr->seen_messages_count == 0)
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1039 return FALSE;
442
2a25bf21557b SEARCH SEEN, UNSEEN, DELETED, UNDELETED, RECENT, UNRECENT optimizations.
Timo Sirainen <tss@iki.fi>
parents: 441
diff changeset
1040
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1041 if (hdr->seen_messages_count == hdr->messages_count) {
1844
4c2678dd65f1 Some optimizations to messageset handling in search.
Timo Sirainen <tss@iki.fi>
parents: 1782
diff changeset
1042 /* UNSEEN with all seen? */
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
1043 if (args->match_not)
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1044 return FALSE;
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
1045 } else if (args->match_not) {
1844
4c2678dd65f1 Some optimizations to messageset handling in search.
Timo Sirainen <tss@iki.fi>
parents: 1782
diff changeset
1046 /* UNSEEN with lowwater limiting */
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1047 search_limit_lowwater(ctx,
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1048 hdr->first_unseen_uid_lowwater, seq1);
1844
4c2678dd65f1 Some optimizations to messageset handling in search.
Timo Sirainen <tss@iki.fi>
parents: 1782
diff changeset
1049 }
6962
659e4a606aae Replaced SEARCH_<flag> with a generic SEARCH_FLAGS. Use value.size for
Timo Sirainen <tss@iki.fi>
parents: 6940
diff changeset
1050 }
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1051 if ((args->value.flags & MAIL_DELETED) != 0 &&
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1052 (pvt_flags_mask & MAIL_DELETED) == 0) {
442
2a25bf21557b SEARCH SEEN, UNSEEN, DELETED, UNDELETED, RECENT, UNRECENT optimizations.
Timo Sirainen <tss@iki.fi>
parents: 441
diff changeset
1053 /* DELETED with 0 deleted? */
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
1054 if (!args->match_not &&
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1055 hdr->deleted_messages_count == 0)
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1056 return FALSE;
442
2a25bf21557b SEARCH SEEN, UNSEEN, DELETED, UNDELETED, RECENT, UNRECENT optimizations.
Timo Sirainen <tss@iki.fi>
parents: 441
diff changeset
1057
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1058 if (hdr->deleted_messages_count == hdr->messages_count) {
1844
4c2678dd65f1 Some optimizations to messageset handling in search.
Timo Sirainen <tss@iki.fi>
parents: 1782
diff changeset
1059 /* UNDELETED with all deleted? */
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
1060 if (args->match_not)
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1061 return FALSE;
13093
252f64964403 lib-storage: Renamed mail_search_arg.not to match_not.
Timo Sirainen <tss@iki.fi>
parents: 13086
diff changeset
1062 } else if (!args->match_not) {
1844
4c2678dd65f1 Some optimizations to messageset handling in search.
Timo Sirainen <tss@iki.fi>
parents: 1782
diff changeset
1063 /* DELETED with lowwater limiting */
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1064 search_limit_lowwater(ctx,
14738
c3a781ef0aeb lib-storage: Fixes to handling private message flag indexes.
Timo Sirainen <tss@iki.fi>
parents: 14711
diff changeset
1065 hdr->first_deleted_uid_lowwater, seq1);
1844
4c2678dd65f1 Some optimizations to messageset handling in search.
Timo Sirainen <tss@iki.fi>
parents: 1782
diff changeset
1066 }
442
2a25bf21557b SEARCH SEEN, UNSEEN, DELETED, UNDELETED, RECENT, UNRECENT optimizations.
Timo Sirainen <tss@iki.fi>
parents: 441
diff changeset
1067 }
962
3b139ccc1858 SEARCH didn't properly complain about invalid messagesets. high:low never
Timo Sirainen <tss@iki.fi>
parents: 951
diff changeset
1068 }
3b139ccc1858 SEARCH didn't properly complain about invalid messagesets. high:low never
Timo Sirainen <tss@iki.fi>
parents: 951
diff changeset
1069
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1070 return *seq1 <= *seq2;
327
276b7a53c264 Modify log fixes. STORE and SEARCH didn't handle properly message sequence
Timo Sirainen <tss@iki.fi>
parents: 317
diff changeset
1071 }
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1072
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1073 static void search_get_seqset(struct index_search_context *ctx,
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
1074 unsigned int messages_count,
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1075 struct mail_search_arg *args)
327
276b7a53c264 Modify log fixes. STORE and SEARCH didn't handle properly message sequence
Timo Sirainen <tss@iki.fi>
parents: 317
diff changeset
1076 {
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
1077 if (messages_count == 0) {
2899
1a05764af1e0 UID FETCH * gave error message with empty mailbox.
Timo Sirainen <tss@iki.fi>
parents: 2892
diff changeset
1078 /* no messages, don't check sequence ranges. although we could
1a05764af1e0 UID FETCH * gave error message with empty mailbox.
Timo Sirainen <tss@iki.fi>
parents: 2892
diff changeset
1079 give error message then for FETCH, we shouldn't do it for
1a05764af1e0 UID FETCH * gave error message with empty mailbox.
Timo Sirainen <tss@iki.fi>
parents: 2892
diff changeset
1080 UID FETCH. */
1985
3ccd4eb52ee4 don't crash when requesting messages from empty mailbox
Timo Sirainen <tss@iki.fi>
parents: 1958
diff changeset
1081 ctx->seq1 = 1;
3ccd4eb52ee4 don't crash when requesting messages from empty mailbox
Timo Sirainen <tss@iki.fi>
parents: 1958
diff changeset
1082 ctx->seq2 = 0;
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1083 return;
1985
3ccd4eb52ee4 don't crash when requesting messages from empty mailbox
Timo Sirainen <tss@iki.fi>
parents: 1958
diff changeset
1084 }
3ccd4eb52ee4 don't crash when requesting messages from empty mailbox
Timo Sirainen <tss@iki.fi>
parents: 1958
diff changeset
1085
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
1086 ctx->seq1 = 1;
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
1087 ctx->seq2 = messages_count;
4431
2461061327ed NOT condition wasn't handled correctly in SEARCH.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4303
diff changeset
1088
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
1089 search_parse_msgset_args(messages_count, args, &ctx->seq1, &ctx->seq2);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1090 if (ctx->seq1 == 0) {
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1091 ctx->seq1 = 1;
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
1092 ctx->seq2 = messages_count;
327
276b7a53c264 Modify log fixes. STORE and SEARCH didn't handle properly message sequence
Timo Sirainen <tss@iki.fi>
parents: 317
diff changeset
1093 }
4438
00db36e1b734 Don't crash when different search conditions reduce the search range so that
Timo Sirainen <tss@iki.fi>
parents: 4432
diff changeset
1094 if (ctx->seq1 > ctx->seq2) {
4432
c13d8fe698f8 Don't crash if seqset restrictions caused no messages to be searched.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4431
diff changeset
1095 /* no matches */
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1096 return;
4432
c13d8fe698f8 Don't crash if seqset restrictions caused no messages to be searched.
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4431
diff changeset
1097 }
327
276b7a53c264 Modify log fixes. STORE and SEARCH didn't handle properly message sequence
Timo Sirainen <tss@iki.fi>
parents: 317
diff changeset
1098
16389
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1099 /* See if this search query can never match based on data in index's
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1100 header. We'll scan only the root level args, which is usually
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1101 enough. */
3b6ec8db4166 lib-storage: Optimize SEARCH_MODSEQ query if it's higher than HIGHESTMODSEQ.
Timo Sirainen <tss@iki.fi>
parents: 16306
diff changeset
1102 if (!search_limit_by_hdr(ctx, args, &ctx->seq1, &ctx->seq2)) {
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1103 /* no matches */
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1104 ctx->seq1 = 1;
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1105 ctx->seq2 = 0;
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1106 }
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1107 }
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1108
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1109 static int search_build_subthread(struct mail_thread_iterate_context *iter,
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1110 ARRAY_TYPE(seq_range) *uids)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1111 {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1112 struct mail_thread_iterate_context *child_iter;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1113 const struct mail_thread_child_node *node;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1114 int ret = 0;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1115
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1116 while ((node = mail_thread_iterate_next(iter, &child_iter)) != NULL) {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1117 if (child_iter != NULL) {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1118 if (search_build_subthread(child_iter, uids) < 0)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1119 ret = -1;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1120 }
14676
69ba6977bed8 seq_range_array_add() API changed. Added other functions to provide the less common use cases.
Timo Sirainen <tss@iki.fi>
parents: 14629
diff changeset
1121 seq_range_array_add(uids, node->uid);
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1122 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1123 if (mail_thread_iterate_deinit(&iter) < 0)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1124 ret = -1;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1125 return ret;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1126 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1127
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1128 static int search_build_inthread_result(struct index_search_context *ctx,
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1129 struct mail_search_arg *arg)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1130 {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1131 struct mail_thread_iterate_context *iter, *child_iter;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1132 const struct mail_thread_child_node *node;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1133 const ARRAY_TYPE(seq_range) *search_uids;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1134 ARRAY_TYPE(seq_range) thread_uids;
8074
05565bfadc6b INTHREAD search may have failed randomly.
Timo Sirainen <tss@iki.fi>
parents: 8011
diff changeset
1135 int ret = 0;
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1136
9305
34e4af1f5009 Search: Added an assert.
Timo Sirainen <tss@iki.fi>
parents: 9174
diff changeset
1137 /* mail_search_args_init() must have been called by now */
18722
e6513dd519d5 lib-storage: Cleanup - separate search arg values that are set by mail_search_init()
Timo Sirainen <tss@iki.fi>
parents: 18147
diff changeset
1138 i_assert(arg->initialized.search_args != NULL);
9305
34e4af1f5009 Search: Added an assert.
Timo Sirainen <tss@iki.fi>
parents: 9174
diff changeset
1139
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1140 p_array_init(&arg->value.seqset, ctx->mail_ctx.args->pool, 64);
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1141 if (mailbox_search_result_build(ctx->mail_ctx.transaction,
18722
e6513dd519d5 lib-storage: Cleanup - separate search arg values that are set by mail_search_init()
Timo Sirainen <tss@iki.fi>
parents: 18147
diff changeset
1142 arg->initialized.search_args,
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1143 MAILBOX_SEARCH_RESULT_FLAG_UPDATE |
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1144 MAILBOX_SEARCH_RESULT_FLAG_QUEUE_SYNC,
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1145 &arg->value.search_result) < 0)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1146 return -1;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1147 if (ctx->thread_ctx == NULL) {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1148 /* failed earlier */
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1149 return -1;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1150 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1151
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1152 search_uids = mailbox_search_result_get(arg->value.search_result);
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1153 if (array_count(search_uids) == 0) {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1154 /* search found nothing - no threads can match */
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1155 return 0;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1156 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1157
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1158 t_array_init(&thread_uids, 128);
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1159 iter = mail_thread_iterate_init(ctx->thread_ctx,
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1160 arg->value.thread_type, FALSE);
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1161 while ((node = mail_thread_iterate_next(iter, &child_iter)) != NULL) {
14676
69ba6977bed8 seq_range_array_add() API changed. Added other functions to provide the less common use cases.
Timo Sirainen <tss@iki.fi>
parents: 14629
diff changeset
1162 seq_range_array_add(&thread_uids, node->uid);
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1163 if (child_iter != NULL) {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1164 if (search_build_subthread(child_iter,
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1165 &thread_uids) < 0)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1166 ret = -1;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1167 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1168 if (seq_range_array_have_common(&thread_uids, search_uids)) {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1169 /* yes, we want this thread */
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1170 seq_range_array_merge(&arg->value.seqset, &thread_uids);
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1171 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1172 array_clear(&thread_uids);
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1173 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1174 if (mail_thread_iterate_deinit(&iter) < 0)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1175 ret = -1;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1176 return ret;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1177 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1178
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1179 static int search_build_inthreads(struct index_search_context *ctx,
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1180 struct mail_search_arg *arg)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1181 {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1182 int ret = 0;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1183
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1184 for (; arg != NULL; arg = arg->next) {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1185 switch (arg->type) {
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1186 case SEARCH_OR:
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1187 case SEARCH_SUB:
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1188 if (search_build_inthreads(ctx, arg->value.subargs) < 0)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1189 ret = -1;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1190 break;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1191 case SEARCH_INTHREAD:
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1192 if (search_build_inthread_result(ctx, arg) < 0)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1193 ret = -1;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1194 break;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1195 default:
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1196 break;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1197 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1198 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1199 return ret;
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1200 }
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1201
12611
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1202 static void
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1203 wanted_sort_fields_get(struct mailbox *box,
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1204 const enum mail_sort_type *sort_program,
12885
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1205 struct mailbox_header_lookup_ctx *wanted_headers,
12611
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1206 enum mail_fetch_field *wanted_fields_r,
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1207 struct mailbox_header_lookup_ctx **headers_ctx_r)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1208 {
12885
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1209 ARRAY_TYPE(const_string) headers;
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1210 const char *header;
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1211 unsigned int i;
12611
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1212
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1213 *wanted_fields_r = 0;
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1214 *headers_ctx_r = NULL;
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1215
12885
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1216 t_array_init(&headers, 8);
12886
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1217 for (i = 0; sort_program[i] != MAIL_SORT_END; i++) {
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1218 header = NULL;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1219
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1220 switch (sort_program[i] & MAIL_SORT_MASK) {
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1221 case MAIL_SORT_ARRIVAL:
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1222 *wanted_fields_r |= MAIL_FETCH_RECEIVED_DATE;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1223 break;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1224 case MAIL_SORT_CC:
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1225 header = "Cc";
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1226 break;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1227 case MAIL_SORT_DATE:
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1228 *wanted_fields_r |= MAIL_FETCH_DATE;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1229 break;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1230 case MAIL_SORT_FROM:
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1231 header = "From";
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1232 break;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1233 case MAIL_SORT_SIZE:
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1234 *wanted_fields_r |= MAIL_FETCH_VIRTUAL_SIZE;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1235 break;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1236 case MAIL_SORT_SUBJECT:
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1237 header = "Subject";
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1238 break;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1239 case MAIL_SORT_TO:
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1240 header = "To";
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1241 break;
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1242 }
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1243 if (header != NULL)
1ff458ba2e4b lib-storage: Request sort program's all fields, not just the first one.
Timo Sirainen <tss@iki.fi>
parents: 12885
diff changeset
1244 array_append(&headers, &header, 1);
12611
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1245 }
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1246
12885
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1247 if (wanted_headers != NULL) {
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1248 for (i = 0; wanted_headers->name[i] != NULL; i++)
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1249 array_append(&headers, &wanted_headers->name[i], 1);
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1250 }
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1251
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1252 if (array_count(&headers) > 0) {
14686
9ff19c1d5f69 Added array_append_zero() to write a zero-filled record to an array.
Timo Sirainen <tss@iki.fi>
parents: 14682
diff changeset
1253 array_append_zero(&headers);
12885
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1254 *headers_ctx_r = mailbox_header_lookup_init(box,
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1255 array_idx(&headers, 0));
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1256 }
12611
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1257 }
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1258
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1259 struct mail_search_context *
10946
0cf62ad4c7bf lib-storage: Moved index transaction/view from index_transaction to mailbox_transaction.
Timo Sirainen <tss@iki.fi>
parents: 10660
diff changeset
1260 index_storage_search_init(struct mailbox_transaction_context *t,
7642
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1261 struct mail_search_args *args,
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1262 const enum mail_sort_type *sort_program,
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1263 enum mail_fetch_field wanted_fields,
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1264 struct mailbox_header_lookup_ctx *wanted_headers)
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1265 {
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1266 struct index_search_context *ctx;
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
1267 struct mailbox_status status;
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1268
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1269 ctx = i_new(struct index_search_context, 1);
10946
0cf62ad4c7bf lib-storage: Moved index transaction/view from index_transaction to mailbox_transaction.
Timo Sirainen <tss@iki.fi>
parents: 10660
diff changeset
1270 ctx->mail_ctx.transaction = t;
15053
c976a9c01613 Replaced "decomposed titlecase" conversions with more generic normalizer function.
Timo Sirainen <tss@iki.fi>
parents: 14921
diff changeset
1271 ctx->mail_ctx.normalizer = t->box->storage->user->default_normalizer;
10946
0cf62ad4c7bf lib-storage: Moved index transaction/view from index_transaction to mailbox_transaction.
Timo Sirainen <tss@iki.fi>
parents: 10660
diff changeset
1272 ctx->box = t->box;
0cf62ad4c7bf lib-storage: Moved index transaction/view from index_transaction to mailbox_transaction.
Timo Sirainen <tss@iki.fi>
parents: 10660
diff changeset
1273 ctx->view = t->view;
4607
71b3570946f8 Moved some search parameters to public struct mail_search_context and added
Timo Sirainen <tss@iki.fi>
parents: 4439
diff changeset
1274 ctx->mail_ctx.args = args;
10946
0cf62ad4c7bf lib-storage: Moved index transaction/view from index_transaction to mailbox_transaction.
Timo Sirainen <tss@iki.fi>
parents: 10660
diff changeset
1275 ctx->mail_ctx.sort_program = index_sort_program_init(t, sort_program);
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1276
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1277 ctx->max_mails = t->box->storage->set->mail_prefetch_count + 1;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1278 if (ctx->max_mails == 0)
15904
d3cf06639864 Replaced all -1U and (unsigned int)-1 with UINT_MAX.
Timo Sirainen <tss@iki.fi>
parents: 15715
diff changeset
1279 ctx->max_mails = UINT_MAX;
9164
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1280 ctx->next_time_check_cost = SEARCH_INITIAL_MAX_COST;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1281 if (gettimeofday(&ctx->last_nonblock_timeval, NULL) < 0)
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1282 i_fatal("gettimeofday() failed: %m");
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1283
12550
2c299c0e3bc8 lib-storage: Moved some items from mailbox_get_status() to a new mailbox_get_metadata().
Timo Sirainen <tss@iki.fi>
parents: 12254
diff changeset
1284 mailbox_get_open_status(t->box, STATUS_MESSAGES, &status);
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
1285 ctx->mail_ctx.progress_max = status.messages;
8507
f323bf2465bd Give more correct "* OK searched n%" notifications with fts and virtual mailboxes.
Timo Sirainen <tss@iki.fi>
parents: 8497
diff changeset
1286
7647
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1287 i_array_init(&ctx->mail_ctx.results, 5);
4607
71b3570946f8 Moved some search parameters to public struct mail_search_context and added
Timo Sirainen <tss@iki.fi>
parents: 4439
diff changeset
1288 array_create(&ctx->mail_ctx.module_contexts, default_pool,
71b3570946f8 Moved some search parameters to public struct mail_search_context and added
Timo Sirainen <tss@iki.fi>
parents: 4439
diff changeset
1289 sizeof(void *), 5);
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1290 i_array_init(&ctx->mails, ctx->max_mails);
4607
71b3570946f8 Moved some search parameters to public struct mail_search_context and added
Timo Sirainen <tss@iki.fi>
parents: 4439
diff changeset
1291
7642
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1292 mail_search_args_reset(ctx->mail_ctx.args->args, TRUE);
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1293 if (args->have_inthreads) {
10946
0cf62ad4c7bf lib-storage: Moved index transaction/view from index_transaction to mailbox_transaction.
Timo Sirainen <tss@iki.fi>
parents: 10660
diff changeset
1294 if (mail_thread_init(t->box, NULL, &ctx->thread_ctx) < 0)
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
1295 search_set_failed(ctx);
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1296 if (search_build_inthreads(ctx, args->args) < 0)
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
1297 search_set_failed(ctx);
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1298 }
1844
4c2678dd65f1 Some optimizations to messageset handling in search.
Timo Sirainen <tss@iki.fi>
parents: 1782
diff changeset
1299
12611
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1300 if (sort_program != NULL) {
12885
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1301 wanted_sort_fields_get(ctx->box, sort_program, wanted_headers,
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1302 &ctx->mail_ctx.wanted_fields,
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1303 &ctx->mail_ctx.wanted_headers);
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1304 } else if (wanted_headers != NULL) {
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1305 ctx->mail_ctx.wanted_headers = wanted_headers;
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1306 mailbox_header_lookup_ref(wanted_headers);
12611
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1307 }
12885
2c04c3dcd544 lib-storage: Removed struct mail_private.extra_* fields.
Timo Sirainen <tss@iki.fi>
parents: 12877
diff changeset
1308 ctx->mail_ctx.wanted_fields |= wanted_fields;
12611
16a541057e88 lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code.
Timo Sirainen <tss@iki.fi>
parents: 12608
diff changeset
1309
10643
e0b6f739510d lib-storage: Use mailbox_status() to get number of messages in search code.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
1310 search_get_seqset(ctx, status.messages, args->args);
7642
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1311 (void)mail_search_args_foreach(args->args, search_init_arg, ctx);
6275
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1312
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all
Timo Sirainen <tss@iki.fi>
parents: 5613
diff changeset
1313 /* Need to reset results for match_always cases */
7642
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1314 mail_search_args_reset(ctx->mail_ctx.args->args, FALSE);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1315 return &ctx->mail_ctx;
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1316 }
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1317
14629
c93ca5e46a8a Marked functions parameters that are allowed to be NULL. Some APIs were also changed.
Timo Sirainen <tss@iki.fi>
parents: 14319
diff changeset
1318 static void ATTR_NULL(2)
c93ca5e46a8a Marked functions parameters that are allowed to be NULL. Some APIs were also changed.
Timo Sirainen <tss@iki.fi>
parents: 14319
diff changeset
1319 search_arg_deinit(struct mail_search_arg *arg,
22511
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1320 struct index_search_context *ctx)
5527
3b25d89caa93 memory leak fixes
Timo Sirainen <tss@iki.fi>
parents: 5525
diff changeset
1321 {
22511
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1322 switch (arg->type) {
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1323 case SEARCH_MIMEPART:
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1324 index_search_mime_arg_deinit(arg, ctx);
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1325 break;
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1326 default:
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1327 if (arg->context != NULL) {
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1328 struct message_search_context *search_ctx = arg->context;
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1329 message_search_deinit(&search_ctx);
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1330 arg->context = NULL;
f0694e6eda8d lib-storage: index: Made MIME FILENAME search criterion match case-insensitively.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 22005
diff changeset
1331 }
5527
3b25d89caa93 memory leak fixes
Timo Sirainen <tss@iki.fi>
parents: 5525
diff changeset
1332 }
3b25d89caa93 memory leak fixes
Timo Sirainen <tss@iki.fi>
parents: 5525
diff changeset
1333 }
3b25d89caa93 memory leak fixes
Timo Sirainen <tss@iki.fi>
parents: 5525
diff changeset
1334
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1335 int index_storage_search_deinit(struct mail_search_context *_ctx)
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1336 {
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1337 struct index_search_context *ctx = (struct index_search_context *)_ctx;
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1338 struct mail **mailp;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1339 int ret;
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1340
12871
d399d5c65aca Changed header/body searching to do search key charset translation earlier.
Timo Sirainen <tss@iki.fi>
parents: 12870
diff changeset
1341 ret = ctx->failed ? -1 : 0;
327
276b7a53c264 Modify log fixes. STORE and SEARCH didn't handle properly message sequence
Timo Sirainen <tss@iki.fi>
parents: 317
diff changeset
1342
7642
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1343 mail_search_args_reset(ctx->mail_ctx.args->args, FALSE);
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1344 (void)mail_search_args_foreach(ctx->mail_ctx.args->args,
15079
925d4a890a9b Fixed compiling with OSes where NULL isn't defined as void pointer (e.g. Solaris).
Timo Sirainen <tss@iki.fi>
parents: 15055
diff changeset
1345 search_arg_deinit, ctx);
5527
3b25d89caa93 memory leak fixes
Timo Sirainen <tss@iki.fi>
parents: 5525
diff changeset
1346
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1347 if (ctx->mail_ctx.wanted_headers != NULL)
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1348 mailbox_header_lookup_unref(&ctx->mail_ctx.wanted_headers);
21599
a90f51ba2e9c lib-storage: Fix error handling when sorting mails.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21390
diff changeset
1349 if (ctx->mail_ctx.sort_program != NULL) {
a90f51ba2e9c lib-storage: Fix error handling when sorting mails.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21390
diff changeset
1350 if (index_sort_program_deinit(&ctx->mail_ctx.sort_program) < 0)
a90f51ba2e9c lib-storage: Fix error handling when sorting mails.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21390
diff changeset
1351 ret = -1;
a90f51ba2e9c lib-storage: Fix error handling when sorting mails.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21390
diff changeset
1352 }
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1353 if (ctx->thread_ctx != NULL)
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1354 mail_thread_deinit(&ctx->thread_ctx);
7647
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1355 array_free(&ctx->mail_ctx.results);
4607
71b3570946f8 Moved some search parameters to public struct mail_search_context and added
Timo Sirainen <tss@iki.fi>
parents: 4439
diff changeset
1356 array_free(&ctx->mail_ctx.module_contexts);
12870
1fbd59c48414 lib-storage: Changed mailbox_search_*() API to return pointer struct mail.
Timo Sirainen <tss@iki.fi>
parents: 12860
diff changeset
1357
13251
390f69281fb8 lib-storage: Fixed assert-crash caused by previous stats change.
Timo Sirainen <tss@iki.fi>
parents: 13250
diff changeset
1358 array_foreach_modifiable(&ctx->mails, mailp) {
390f69281fb8 lib-storage: Fixed assert-crash caused by previous stats change.
Timo Sirainen <tss@iki.fi>
parents: 13250
diff changeset
1359 struct index_mail *imail = (struct index_mail *)*mailp;
390f69281fb8 lib-storage: Fixed assert-crash caused by previous stats change.
Timo Sirainen <tss@iki.fi>
parents: 13250
diff changeset
1360
390f69281fb8 lib-storage: Fixed assert-crash caused by previous stats change.
Timo Sirainen <tss@iki.fi>
parents: 13250
diff changeset
1361 imail->search_mail = FALSE;
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1362 mail_free(mailp);
13251
390f69281fb8 lib-storage: Fixed assert-crash caused by previous stats change.
Timo Sirainen <tss@iki.fi>
parents: 13250
diff changeset
1363 }
21601
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
1364
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
1365 if (ctx->failed)
15e700c083c0 lib-storage: Fix error handling when searching mails
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21600
diff changeset
1366 mail_storage_last_error_pop(ctx->box->storage);
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1367 array_free(&ctx->mails);
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1368 i_free(ctx);
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1369 return ret;
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1370 }
327
276b7a53c264 Modify log fixes. STORE and SEARCH didn't handle properly message sequence
Timo Sirainen <tss@iki.fi>
parents: 317
diff changeset
1371
13250
b7a6573dacca lib-storage: Moved stats_* from mail to transaction.
Timo Sirainen <tss@iki.fi>
parents: 13150
diff changeset
1372 static unsigned long long
b7a6573dacca lib-storage: Moved stats_* from mail to transaction.
Timo Sirainen <tss@iki.fi>
parents: 13150
diff changeset
1373 search_get_cost(struct mailbox_transaction_context *trans)
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
1374 {
13289
4317429862e1 lib-storage: Moved all transaction stats_* to struct mailbox_transaction_stats.
Timo Sirainen <tss@iki.fi>
parents: 13287
diff changeset
1375 return trans->stats.open_lookup_count * SEARCH_COST_DENTRY +
4317429862e1 lib-storage: Moved all transaction stats_* to struct mailbox_transaction_stats.
Timo Sirainen <tss@iki.fi>
parents: 13287
diff changeset
1376 trans->stats.stat_lookup_count * SEARCH_COST_DENTRY +
4317429862e1 lib-storage: Moved all transaction stats_* to struct mailbox_transaction_stats.
Timo Sirainen <tss@iki.fi>
parents: 13287
diff changeset
1377 trans->stats.fstat_lookup_count * SEARCH_COST_ATTR +
4317429862e1 lib-storage: Moved all transaction stats_* to struct mailbox_transaction_stats.
Timo Sirainen <tss@iki.fi>
parents: 13287
diff changeset
1378 trans->stats.cache_hit_count * SEARCH_COST_CACHE +
4317429862e1 lib-storage: Moved all transaction stats_* to struct mailbox_transaction_stats.
Timo Sirainen <tss@iki.fi>
parents: 13287
diff changeset
1379 trans->stats.files_read_count * SEARCH_COST_FILES_READ +
4317429862e1 lib-storage: Moved all transaction stats_* to struct mailbox_transaction_stats.
Timo Sirainen <tss@iki.fi>
parents: 13287
diff changeset
1380 (trans->stats.files_read_bytes/1024) * SEARCH_COST_KBYTE;
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
1381 }
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
1382
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1383 static int search_match_once(struct index_search_context *ctx)
4949
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
1384 {
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1385 int ret;
4949
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
1386
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1387 ret = mail_search_args_foreach(ctx->mail_ctx.args->args,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1388 search_cached_arg, ctx);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1389 if (ret < 0)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1390 ret = search_arg_match_text(ctx->mail_ctx.args->args, ctx);
21641
e9a0ccfe9b67 Partially implemented IMAP SEARCH=X-MIMEPART capability.
Stephan Bosch <stephan@dovecot.fi>
parents: 21606
diff changeset
1391 if (ret < 0)
e9a0ccfe9b67 Partially implemented IMAP SEARCH=X-MIMEPART capability.
Stephan Bosch <stephan@dovecot.fi>
parents: 21606
diff changeset
1392 ret = index_search_mime_arg_match(ctx->mail_ctx.args->args, ctx);
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1393 return ret;
4949
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
1394 }
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
1395
7647
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1396 static bool search_arg_is_static(struct mail_search_arg *arg)
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1397 {
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1398 struct mail_search_arg *subarg;
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1399
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1400 switch (arg->type) {
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1401 case SEARCH_OR:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1402 case SEARCH_SUB:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1403 /* they're static only if all subargs are static */
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1404 subarg = arg->value.subargs;
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1405 for (; subarg != NULL; subarg = subarg->next) {
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1406 if (!search_arg_is_static(subarg))
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1407 return FALSE;
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1408 }
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1409 return TRUE;
7908
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
1410 case SEARCH_SEQSET:
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
1411 /* changes between syncs, but we can't really handle this
1e69c84a1e5a Search code cleanups and minor optimizations.
Timo Sirainen <tss@iki.fi>
parents: 7828
diff changeset
1412 currently. seqsets should be converted to uidsets first. */
7647
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1413 case SEARCH_FLAGS:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1414 case SEARCH_KEYWORDS:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1415 case SEARCH_MODSEQ:
7909
bf9c51edbc66 Added non-optimized support for SEARCH INTHREAD.
Timo Sirainen <tss@iki.fi>
parents: 7908
diff changeset
1416 case SEARCH_INTHREAD:
7647
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1417 break;
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1418 case SEARCH_ALL:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1419 case SEARCH_UIDSET:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1420 case SEARCH_BEFORE:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1421 case SEARCH_ON:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1422 case SEARCH_SINCE:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1423 case SEARCH_SMALLER:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1424 case SEARCH_LARGER:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1425 case SEARCH_HEADER:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1426 case SEARCH_HEADER_ADDRESS:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1427 case SEARCH_HEADER_COMPRESS_LWSP:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1428 case SEARCH_BODY:
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1429 case SEARCH_TEXT:
8077
6d51328896d6 Added the concept of Global UIDs that are preserved across copies.
Timo Sirainen <tss@iki.fi>
parents: 8074
diff changeset
1430 case SEARCH_GUID:
7910
c1bbdc2b262e FETCH X-MAILBOX and SEARCH X-MAILBOX can be used with virtual mailboxes to
Timo Sirainen <tss@iki.fi>
parents: 7909
diff changeset
1431 case SEARCH_MAILBOX:
11232
389a251c9cfe lib-storage: Added support for searching with mailbox GUID.
Timo Sirainen <tss@iki.fi>
parents: 11197
diff changeset
1432 case SEARCH_MAILBOX_GUID:
11197
3c9770bae39e lib-storage: Added MAILBOX_GLOB search arg. Query builders now use it instead of MAILBOX.
Timo Sirainen <tss@iki.fi>
parents: 11141
diff changeset
1433 case SEARCH_MAILBOX_GLOB:
17949
98b5ccc539b5 lib-storage: Added X-REAL-UID search parameter.
Timo Sirainen <tss@iki.fi>
parents: 17817
diff changeset
1434 case SEARCH_REAL_UID:
21641
e9a0ccfe9b67 Partially implemented IMAP SEARCH=X-MIMEPART capability.
Stephan Bosch <stephan@dovecot.fi>
parents: 21606
diff changeset
1435 case SEARCH_MIMEPART:
7647
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1436 return TRUE;
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1437 }
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1438 return FALSE;
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1439 }
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1440
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1441 static void search_set_static_matches(struct mail_search_arg *arg)
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1442 {
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1443 for (; arg != NULL; arg = arg->next) {
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1444 if (search_arg_is_static(arg))
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1445 arg->result = 1;
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1446 }
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1447 }
879208fdc7e3 Implemented CONTEXT=SEARCH extension.
Timo Sirainen <tss@iki.fi>
parents: 7642
diff changeset
1448
7671
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1449 static bool search_has_static_nonmatches(struct mail_search_arg *arg)
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1450 {
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1451 for (; arg != NULL; arg = arg->next) {
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1452 if (arg->result == 0 && search_arg_is_static(arg))
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1453 return TRUE;
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1454 }
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1455 return FALSE;
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1456 }
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1457
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1458 static void search_match_finish(struct index_search_context *ctx, int match)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1459 {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1460 if (match == 0 &&
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1461 search_has_static_nonmatches(ctx->mail_ctx.args->args)) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1462 /* if there are saved search results remember
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1463 that this message never matches */
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1464 mailbox_search_results_never(&ctx->mail_ctx,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1465 ctx->cur_mail->uid);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1466 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1467 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1468
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1469 static int search_match_next(struct index_search_context *ctx)
9164
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1470 {
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1471 static enum mail_lookup_abort cache_lookups[] = {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1472 MAIL_LOOKUP_ABORT_NOT_IN_CACHE,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1473 MAIL_LOOKUP_ABORT_READ_MAIL,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1474 MAIL_LOOKUP_ABORT_NEVER
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1475 };
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1476 unsigned int i, n = N_ELEMENTS(cache_lookups);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1477 int ret = -1;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1478
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1479 if (ctx->have_mailbox_args) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1480 /* check that the mailbox name matches.
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1481 this makes sense only with virtual mailboxes. */
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1482 ret = mail_search_args_foreach(ctx->mail_ctx.args->args,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1483 search_mailbox_arg, ctx);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1484 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1485
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1486 /* avoid doing extra work for as long as possible */
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1487 if (ctx->max_mails > 1) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1488 /* we're doing prefetching. if we have to read the mail,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1489 do a prefetch first and the final search later */
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1490 n--;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1491 }
21650
5fb623020110 lib-storage: Add asserts to make sure lookup_abort is being used correctly.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21641
diff changeset
1492
5fb623020110 lib-storage: Add asserts to make sure lookup_abort is being used correctly.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21641
diff changeset
1493 i_assert(ctx->cur_mail->lookup_abort == MAIL_LOOKUP_ABORT_NEVER);
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1494 for (i = 0; i < n && ret < 0; i++) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1495 ctx->cur_mail->lookup_abort = cache_lookups[i];
16306
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1496 T_BEGIN {
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1497 ret = search_match_once(ctx);
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1498 } T_END;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1499 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1500 ctx->cur_mail->lookup_abort = MAIL_LOOKUP_ABORT_NEVER;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1501 search_match_finish(ctx, ret);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1502 return ret;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1503 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1504
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1505 static void index_storage_search_notify(struct mailbox *box,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1506 struct index_search_context *ctx)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1507 {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1508 float percentage;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1509 unsigned int msecs, secs;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1510
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1511 if (ctx->last_notify.tv_sec == 0) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1512 /* set the search time in here, in case a plugin
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1513 already spent some time indexing the mailbox */
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1514 ctx->search_start_time = ioloop_timeval;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1515 } else if (box->storage->callbacks.notify_ok != NULL &&
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1516 !ctx->mail_ctx.progress_hidden) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1517 percentage = ctx->mail_ctx.progress_cur * 100.0 /
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1518 ctx->mail_ctx.progress_max;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1519 msecs = timeval_diff_msecs(&ioloop_timeval,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1520 &ctx->search_start_time);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1521 secs = (msecs / (percentage / 100.0) - msecs) / 1000;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1522
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1523 T_BEGIN {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1524 const char *text;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1525
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1526 text = t_strdup_printf("Searched %d%% of the mailbox, "
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1527 "ETA %d:%02d", (int)percentage,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1528 secs/60, secs%60);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1529 box->storage->callbacks.
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1530 notify_ok(box, text,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1531 box->storage->callback_context);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1532 } T_END;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1533 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1534 ctx->last_notify = ioloop_timeval;
9164
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1535 }
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1536
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1537 static bool search_would_block(struct index_search_context *ctx)
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1538 {
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1539 struct timeval now;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1540 unsigned long long guess_cost;
9769
8716936443db Moved all struct timeval comparing/calculation code to lib/time-util.
Timo Sirainen <tss@iki.fi>
parents: 9704
diff changeset
1541 long long usecs;
9164
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1542 bool ret;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1543
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1544 if (ctx->cost < ctx->next_time_check_cost)
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1545 return FALSE;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1546
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1547 if (gettimeofday(&now, NULL) < 0)
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1548 i_fatal("gettimeofday() failed: %m");
9769
8716936443db Moved all struct timeval comparing/calculation code to lib/time-util.
Timo Sirainen <tss@iki.fi>
parents: 9704
diff changeset
1549
8716936443db Moved all struct timeval comparing/calculation code to lib/time-util.
Timo Sirainen <tss@iki.fi>
parents: 9704
diff changeset
1550 usecs = timeval_diff_usecs(&now, &ctx->last_nonblock_timeval);
8716936443db Moved all struct timeval comparing/calculation code to lib/time-util.
Timo Sirainen <tss@iki.fi>
parents: 9704
diff changeset
1551 if (usecs < 0) {
9164
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1552 /* clock moved backwards. */
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1553 ctx->last_nonblock_timeval = now;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1554 ctx->next_time_check_cost = SEARCH_INITIAL_MAX_COST;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1555 return TRUE;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1556 } else if (usecs < SEARCH_MIN_NONBLOCK_USECS) {
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1557 /* not finished yet. estimate the next time lookup */
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1558 ret = FALSE;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1559 } else {
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1560 /* done, or close enough anyway */
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1561 ctx->last_nonblock_timeval = now;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1562 ret = TRUE;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1563 }
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1564 guess_cost = ctx->cost *
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1565 (SEARCH_MAX_NONBLOCK_USECS / (double)usecs);
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1566 if (usecs < SEARCH_RECALC_MIN_USECS) {
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1567 /* the estimate may not be very good since we spent
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1568 so little time doing this search. don't allow huge changes
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1569 to the guess, but allow anyway large enough so that we can
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1570 move to right direction. */
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1571 if (guess_cost > ctx->next_time_check_cost*3)
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1572 guess_cost = ctx->next_time_check_cost*3;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1573 else if (guess_cost < ctx->next_time_check_cost/3)
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1574 guess_cost = ctx->next_time_check_cost/3;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1575 }
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1576 if (ret)
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1577 ctx->cost = 0;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1578 ctx->next_time_check_cost = guess_cost;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1579 return ret;
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1580 }
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1581
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1582 static int search_more_with_mail(struct index_search_context *ctx,
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1583 struct mail *mail)
988
8028c4dcf38f mail-storage.h interface changes, affects pretty much everything.
Timo Sirainen <tss@iki.fi>
parents: 962
diff changeset
1584 {
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1585 struct mail_search_context *_ctx = &ctx->mail_ctx;
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1586 struct mailbox *box = _ctx->transaction->box;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1587 struct index_mail *imail = (struct index_mail *)mail;
14319
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1588 unsigned long long cost1, cost2;
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1589 int match, ret;
4303
5f03738219a6 Changed mail-storage API to do the mail sorting internally. Optimized it
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4259
diff changeset
1590
9164
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1591 if (search_would_block(ctx)) {
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1592 /* this lookup is useful when a large number of
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1593 messages match */
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1594 return 0;
9164
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1595 }
9b8bf57405c8 Use the new mail_private.stats_* fields to stop non-blocking searches after about 250 ms.
Timo Sirainen <tss@iki.fi>
parents: 9162
diff changeset
1596
4949
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
1597 if (ioloop_time - ctx->last_notify.tv_sec >=
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
1598 SEARCH_NOTIFY_INTERVAL_SECS)
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
1599 index_storage_search_notify(box, ctx);
a959d4208b54 Moved storage callbacks to struct mail_storage. If searching takes over 10
Timo Sirainen <tss@iki.fi>
parents: 4942
diff changeset
1600
13832
5c2ef2374848 lib-storage: Fixed search when mail prefetching is enabled.
Timo Sirainen <tss@iki.fi>
parents: 13601
diff changeset
1601 mail_search_args_reset(_ctx->args->args, FALSE);
5c2ef2374848 lib-storage: Fixed search when mail prefetching is enabled.
Timo Sirainen <tss@iki.fi>
parents: 13601
diff changeset
1602
14319
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1603 cost1 = search_get_cost(mail->transaction);
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1604 ret = -1;
8477
c033b3e2d9b6 mail_storage.search_next_update_seq returns now bool. It was never failing.
Timo Sirainen <tss@iki.fi>
parents: 8411
diff changeset
1605 while (box->v.search_next_update_seq(_ctx)) {
6277
5f66277bbe40 mail_index_lookup*() can't fail anymore. Changed several APIs not to return
Timo Sirainen <tss@iki.fi>
parents: 6276
diff changeset
1606 mail_set_seq(mail, _ctx->seq);
453
0f6fd6802265 Modify log now stores the changes in ranges, so store 1:100 doesn't
Timo Sirainen <tss@iki.fi>
parents: 450
diff changeset
1607
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1608 ctx->cur_mail = mail;
21605
bcd286fcdbad lib-storage: Add mail.access_type
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21601
diff changeset
1609 /* mail's access_type is SEARCH only while using it to process
bcd286fcdbad lib-storage: Add mail.access_type
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21601
diff changeset
1610 the search query. afterwards the mail can still be accessed
bcd286fcdbad lib-storage: Add mail.access_type
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21601
diff changeset
1611 for fetching. */
bcd286fcdbad lib-storage: Add mail.access_type
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21601
diff changeset
1612 ctx->cur_mail->access_type = MAIL_ACCESS_TYPE_SEARCH;
8481
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1613 T_BEGIN {
8477
c033b3e2d9b6 mail_storage.search_next_update_seq returns now bool. It was never failing.
Timo Sirainen <tss@iki.fi>
parents: 8411
diff changeset
1614 match = search_match_next(ctx);
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1615 } T_END;
21605
bcd286fcdbad lib-storage: Add mail.access_type
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21601
diff changeset
1616 ctx->cur_mail->access_type = MAIL_ACCESS_TYPE_DEFAULT;
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1617 ctx->cur_mail = NULL;
8411
abd0ef855a33 Implemented imap-response-codes draft.
Timo Sirainen <tss@iki.fi>
parents: 8146
diff changeset
1618
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1619 i_assert(imail->data.search_results == NULL);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1620 if (match < 0) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1621 /* result isn't known yet, do a prefetch and
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1622 finish later */
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1623 imail->data.search_results =
15055
c436a6c3f7d2 lib-storage: Moved index_mail.data_pool to mail_private.data_pool
Timo Sirainen <tss@iki.fi>
parents: 15053
diff changeset
1624 buffer_create_dynamic(imail->mail.data_pool, 64);
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1625 mail_search_args_result_serialize(_ctx->args,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1626 imail->data.search_results);
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1627 }
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
1628
7671
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1629 mail_search_args_reset(_ctx->args->args, FALSE);
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1630
14319
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1631 if (match != 0) {
18106
2553f3aae9bb lib-storage: Don't fetch wanted_fields for messages that don't match the search query.
Timo Sirainen <tss@iki.fi>
parents: 17996
diff changeset
1632 /* either matched or result is still unknown.
2553f3aae9bb lib-storage: Don't fetch wanted_fields for messages that don't match the search query.
Timo Sirainen <tss@iki.fi>
parents: 17996
diff changeset
1633 anyway we're far enough now that we probably want
2553f3aae9bb lib-storage: Don't fetch wanted_fields for messages that don't match the search query.
Timo Sirainen <tss@iki.fi>
parents: 17996
diff changeset
1634 to update the access_parts. the only problem here is
2553f3aae9bb lib-storage: Don't fetch wanted_fields for messages that don't match the search query.
Timo Sirainen <tss@iki.fi>
parents: 17996
diff changeset
1635 if searching would want fewer access_parts than the
2553f3aae9bb lib-storage: Don't fetch wanted_fields for messages that don't match the search query.
Timo Sirainen <tss@iki.fi>
parents: 17996
diff changeset
1636 fetching part, but that's probably not a big problem
2553f3aae9bb lib-storage: Don't fetch wanted_fields for messages that don't match the search query.
Timo Sirainen <tss@iki.fi>
parents: 17996
diff changeset
1637 usually. */
18128
6dd190bd6dcb lib-storage: Mail prefetching was still not working right.
Timo Sirainen <tss@iki.fi>
parents: 18106
diff changeset
1638 index_mail_update_access_parts_pre(mail);
14319
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1639 ret = 1;
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1640 break;
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1641 }
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1642
18147
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1643 /* non-match */
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1644 if (_ctx->args->stop_on_nonmatch) {
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1645 ret = -1;
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1646 break;
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1647 }
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1648
14319
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1649 cost2 = search_get_cost(mail->transaction);
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1650 ctx->cost += cost2 - cost1;
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1651 cost1 = cost2;
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1652
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1653 if (search_would_block(ctx)) {
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1654 ret = 0;
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1655 break;
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1656 }
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1657 }
14319
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1658 cost2 = search_get_cost(mail->transaction);
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1659 ctx->cost += cost2 - cost1;
00887de4ad32 lib-storage: Update search's cost more widely and less often.
Timo Sirainen <tss@iki.fi>
parents: 14295
diff changeset
1660 return ret;
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1661 }
4303
5f03738219a6 Changed mail-storage API to do the mail sorting internally. Optimized it
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4259
diff changeset
1662
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1663 struct mail *index_search_get_mail(struct index_search_context *ctx)
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1664 {
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1665 struct index_mail *imail;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1666 struct mail *const *mails, *mail;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1667 unsigned int count;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1668
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1669 if (ctx->unused_mail_idx == ctx->max_mails)
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1670 return NULL;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1671
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1672 mails = array_get(&ctx->mails, &count);
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1673 if (ctx->unused_mail_idx < count)
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1674 return mails[ctx->unused_mail_idx];
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1675
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1676 mail = mail_alloc(ctx->mail_ctx.transaction,
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1677 ctx->mail_ctx.wanted_fields,
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1678 ctx->mail_ctx.wanted_headers);
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1679 imail = (struct index_mail *)mail;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1680 imail->search_mail = TRUE;
13250
b7a6573dacca lib-storage: Moved stats_* from mail to transaction.
Timo Sirainen <tss@iki.fi>
parents: 13150
diff changeset
1681 ctx->mail_ctx.transaction->stats_track = TRUE;
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1682
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1683 array_append(&ctx->mails, &mail, 1);
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1684 return mail;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1685 }
4303
5f03738219a6 Changed mail-storage API to do the mail sorting internally. Optimized it
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4259
diff changeset
1686
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1687 static int search_more_with_prefetching(struct index_search_context *ctx,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1688 struct mail **mail_r)
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1689 {
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1690 struct mail *mail, *const *mails;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1691 unsigned int count;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1692 int ret = 0;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1693
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1694 while ((mail = index_search_get_mail(ctx)) != NULL) {
16306
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1695 T_BEGIN {
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1696 ret = search_more_with_mail(ctx, mail);
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1697 } T_END;
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1698 if (ret <= 0)
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1699 break;
14295
0c3cb8976e81 lib-storage: When searching with a sort program, don't prefetch any mails.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
1700
0c3cb8976e81 lib-storage: When searching with a sort program, don't prefetch any mails.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
1701 if (ctx->mail_ctx.sort_program != NULL) {
0c3cb8976e81 lib-storage: When searching with a sort program, don't prefetch any mails.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
1702 /* don't prefetch when using a sort program,
0c3cb8976e81 lib-storage: When searching with a sort program, don't prefetch any mails.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
1703 since the mails' access order will change */
0c3cb8976e81 lib-storage: When searching with a sort program, don't prefetch any mails.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
1704 i_assert(ctx->unused_mail_idx == 0);
0c3cb8976e81 lib-storage: When searching with a sort program, don't prefetch any mails.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
1705 *mail_r = mail;
0c3cb8976e81 lib-storage: When searching with a sort program, don't prefetch any mails.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
1706 return 1;
0c3cb8976e81 lib-storage: When searching with a sort program, don't prefetch any mails.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
1707 }
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1708 if (mail_prefetch(mail) && ctx->unused_mail_idx == 0) {
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1709 /* no prefetching done, return it immediately */
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1710 *mail_r = mail;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1711 return 1;
4303
5f03738219a6 Changed mail-storage API to do the mail sorting internally. Optimized it
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4259
diff changeset
1712 }
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1713 ctx->unused_mail_idx++;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1714 }
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1715
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1716 if (mail != NULL) {
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1717 if (ret == 0) {
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1718 /* wait */
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1719 return 0;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1720 }
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1721 i_assert(ret < 0);
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1722 if (ctx->unused_mail_idx == 0) {
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1723 /* finished */
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1724 return -1;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1725 }
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1726 } else {
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1727 /* prefetch buffer is full. */
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1728 }
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1729
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1730 /* return the next message */
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1731 i_assert(ctx->unused_mail_idx > 0);
4939
ff2272c228cc Dovecot is now able to execute multiple commands at the same time.
Timo Sirainen <tss@iki.fi>
parents: 4907
diff changeset
1732
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1733 mails = array_get(&ctx->mails, &count);
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1734 *mail_r = mails[0];
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1735 if (--ctx->unused_mail_idx > 0) {
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1736 array_delete(&ctx->mails, 0, 1);
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1737 array_append(&ctx->mails, mail_r, 1);
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1738 }
18128
6dd190bd6dcb lib-storage: Mail prefetching was still not working right.
Timo Sirainen <tss@iki.fi>
parents: 18106
diff changeset
1739 index_mail_update_access_parts_post(*mail_r);
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1740 return 1;
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1741 }
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1742
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1743 static bool search_finish_prefetch(struct index_search_context *ctx,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1744 struct index_mail *imail)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1745 {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1746 int ret;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1747
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1748 i_assert(imail->mail.mail.lookup_abort == MAIL_LOOKUP_ABORT_NEVER);
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1749
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1750 ctx->cur_mail = &imail->mail.mail;
21654
2c937505c112 lib-storage: Fix mail.access_type for search prefetches
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21650
diff changeset
1751 ctx->cur_mail->access_type = MAIL_ACCESS_TYPE_SEARCH;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1752 mail_search_args_result_deserialize(ctx->mail_ctx.args,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1753 imail->data.search_results->data,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1754 imail->data.search_results->used);
16306
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1755 T_BEGIN {
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1756 ret = search_match_once(ctx);
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1757 search_match_finish(ctx, ret);
e819374de157 lib-storage: Avoid wasting data stack during searches.
Timo Sirainen <tss@iki.fi>
parents: 15904
diff changeset
1758 } T_END;
21654
2c937505c112 lib-storage: Fix mail.access_type for search prefetches
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 21650
diff changeset
1759 ctx->cur_mail->access_type = MAIL_ACCESS_TYPE_DEFAULT;
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1760 ctx->cur_mail = NULL;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1761 return ret > 0;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1762 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1763
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1764 static int search_more(struct index_search_context *ctx,
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1765 struct mail **mail_r)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1766 {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1767 struct index_mail *imail;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1768 int ret;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1769
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1770 while ((ret = search_more_with_prefetching(ctx, mail_r)) > 0) {
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1771 imail = (struct index_mail *)*mail_r;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1772 if (imail->data.search_results == NULL)
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1773 break;
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
1774
18147
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1775 /* prefetch running - searching wasn't finished yet */
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1776 if (search_finish_prefetch(ctx, imail))
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1777 break;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1778 /* search finished as non-match */
18147
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1779 if (ctx->mail_ctx.args->stop_on_nonmatch) {
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1780 ret = -1;
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1781 break;
14bf136959bc lib-storage: Added "oldestonly" search arg to stop searching after the first non-match.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
1782 }
12877
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1783 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1784 return ret;
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1785 }
51ef5ffea07f lib-storage: Added mail prefetching support for searching.
Timo Sirainen <tss@iki.fi>
parents: 12874
diff changeset
1786
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1787 bool index_storage_search_next_nonblock(struct mail_search_context *_ctx,
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1788 struct mail **mail_r, bool *tryagain_r)
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1789 {
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1790 struct index_search_context *ctx = (struct index_search_context *)_ctx;
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1791 struct mail *mail, *const *mailp;
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1792 uint32_t seq;
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1793 int ret;
4939
ff2272c228cc Dovecot is now able to execute multiple commands at the same time.
Timo Sirainen <tss@iki.fi>
parents: 4907
diff changeset
1794
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1795 *tryagain_r = FALSE;
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1796
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1797 if (_ctx->sort_program == NULL) {
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1798 ret = search_more(ctx, &mail);
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1799 if (ret == 0) {
4939
ff2272c228cc Dovecot is now able to execute multiple commands at the same time.
Timo Sirainen <tss@iki.fi>
parents: 4907
diff changeset
1800 *tryagain_r = TRUE;
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1801 return FALSE;
4939
ff2272c228cc Dovecot is now able to execute multiple commands at the same time.
Timo Sirainen <tss@iki.fi>
parents: 4907
diff changeset
1802 }
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1803 if (ret < 0)
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1804 return FALSE;
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1805 *mail_r = mail;
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1806 return TRUE;
1915
79790750c349 importing new index code. mbox still broken.
Timo Sirainen <tss@iki.fi>
parents: 1845
diff changeset
1807 }
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1808
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1809 if (!ctx->sorted) {
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1810 while ((ret = search_more(ctx, &mail)) > 0)
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1811 index_sort_list_add(_ctx->sort_program, mail);
1672
8920600a8cfc Index cache file rewrite. It's not finished yet and mbox support is
Timo Sirainen <tss@iki.fi>
parents: 1638
diff changeset
1812
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1813 if (ret == 0) {
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1814 *tryagain_r = TRUE;
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1815 return FALSE;
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1816 }
4939
ff2272c228cc Dovecot is now able to execute multiple commands at the same time.
Timo Sirainen <tss@iki.fi>
parents: 4907
diff changeset
1817 /* finished searching the messages. now sort them and start
ff2272c228cc Dovecot is now able to execute multiple commands at the same time.
Timo Sirainen <tss@iki.fi>
parents: 4907
diff changeset
1818 returning the messages. */
4303
5f03738219a6 Changed mail-storage API to do the mail sorting internally. Optimized it
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4259
diff changeset
1819 ctx->sorted = TRUE;
7671
882888286bf5 Saved search results: Remember what messages' static checks never match, so
Timo Sirainen <tss@iki.fi>
parents: 7647
diff changeset
1820 index_sort_list_finish(_ctx->sort_program);
4303
5f03738219a6 Changed mail-storage API to do the mail sorting internally. Optimized it
Timo Sirainen <timo.sirainen@movial.fi>
parents: 4259
diff changeset
1821 }
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1822
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1823 /* everything searched at this point already. just returning
18128
6dd190bd6dcb lib-storage: Mail prefetching was still not working right.
Timo Sirainen <tss@iki.fi>
parents: 18106
diff changeset
1824 matches from sort list. FIXME: we could do prefetching here also. */
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1825 if (!index_sort_list_next(_ctx->sort_program, &seq))
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1826 return FALSE;
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1827
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1828 mailp = array_idx(&ctx->mails, 0);
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1829 mail_set_seq(*mailp, seq);
18128
6dd190bd6dcb lib-storage: Mail prefetching was still not working right.
Timo Sirainen <tss@iki.fi>
parents: 18106
diff changeset
1830 index_mail_update_access_parts_pre(*mailp);
6dd190bd6dcb lib-storage: Mail prefetching was still not working right.
Timo Sirainen <tss@iki.fi>
parents: 18106
diff changeset
1831 index_mail_update_access_parts_post(*mailp);
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1832 *mail_r = *mailp;
12873
b47f17b173f5 lib-storage: Search code cleanup.
Timo Sirainen <tss@iki.fi>
parents: 12871
diff changeset
1833 return TRUE;
0
3b1985cbc908 Initial revision
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
1834 }
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1835
8477
c033b3e2d9b6 mail_storage.search_next_update_seq returns now bool. It was never failing.
Timo Sirainen <tss@iki.fi>
parents: 8411
diff changeset
1836 bool index_storage_search_next_update_seq(struct mail_search_context *_ctx)
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1837 {
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1838 struct index_search_context *ctx = (struct index_search_context *)_ctx;
8481
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1839 uint32_t uid;
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1840 int ret;
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1841
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1842 if (_ctx->seq == 0) {
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1843 /* first time */
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1844 _ctx->seq = ctx->seq1;
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1845 } else {
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1846 _ctx->seq++;
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1847 }
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1848
8481
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1849 if (!ctx->have_seqsets && !ctx->have_index_args &&
8507
f323bf2465bd Give more correct "* OK searched n%" notifications with fts and virtual mailboxes.
Timo Sirainen <tss@iki.fi>
parents: 8497
diff changeset
1850 _ctx->update_result == NULL) {
12874
0461e23ae57c Search supports now prefetching data for returned mails. Dropped imapc's own prefetching.
Timo Sirainen <tss@iki.fi>
parents: 12873
diff changeset
1851 _ctx->progress_cur = _ctx->seq;
8477
c033b3e2d9b6 mail_storage.search_next_update_seq returns now bool. It was never failing.
Timo Sirainen <tss@iki.fi>
parents: 8411
diff changeset
1852 return _ctx->seq <= ctx->seq2;
8507
f323bf2465bd Give more correct "* OK searched n%" notifications with fts and virtual mailboxes.
Timo Sirainen <tss@iki.fi>
parents: 8497
diff changeset
1853 }
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1854
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1855 ret = 0;
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1856 while (_ctx->seq <= ctx->seq2) {
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1857 /* check if the sequence matches */
7642
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1858 ret = mail_search_args_foreach(ctx->mail_ctx.args->args,
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1859 search_seqset_arg, ctx);
8481
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1860 if (ret != 0 && ctx->have_index_args) {
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
1861 /* check if flags/keywords match before anything else
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
1862 is done. mail_set_seq() can be a bit slow. */
7642
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1863 ret = mail_search_args_foreach(ctx->mail_ctx.args->args,
6966
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
1864 search_index_arg, ctx);
7bb6ef75715c Optimize searching flags and keywords.
Timo Sirainen <tss@iki.fi>
parents: 6965
diff changeset
1865 }
8481
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1866 if (ret != 0 && _ctx->update_result != NULL) {
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1867 /* see if this message never matches */
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1868 mail_index_lookup_uid(ctx->view, _ctx->seq, &uid);
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1869 if (seq_range_exists(&_ctx->update_result->never_uids,
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1870 uid))
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1871 ret = 0;
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1872 }
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1873 if (ret != 0)
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1874 break;
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1875
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1876 /* doesn't, try next one */
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1877 _ctx->seq++;
7642
077bb84e9e77 Make mail_search_args an independent structure that can be used for multiple
Timo Sirainen <tss@iki.fi>
parents: 7640
diff changeset
1878 mail_search_args_reset(ctx->mail_ctx.args->args, FALSE);
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1879 }
8481
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1880
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1881 if (ret != 0 && _ctx->update_result != NULL) {
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1882 mail_index_lookup_uid(ctx->view, _ctx->seq, &uid);
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1883 if (seq_range_exists(&_ctx->update_result->uids, uid)) {
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1884 /* we already know that the static data
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1885 matches. mark it as such. */
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1886 search_set_static_matches(_ctx->args->args);
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1887 }
9db582413fef Moved search update result checks to search_next_update_seq().
Timo Sirainen <tss@iki.fi>
parents: 8479
diff changeset
1888 }
8507
f323bf2465bd Give more correct "* OK searched n%" notifications with fts and virtual mailboxes.
Timo Sirainen <tss@iki.fi>
parents: 8497
diff changeset
1889 ctx->mail_ctx.progress_cur = _ctx->seq;
8477
c033b3e2d9b6 mail_storage.search_next_update_seq returns now bool. It was never failing.
Timo Sirainen <tss@iki.fi>
parents: 8411
diff changeset
1890 return ret != 0;
4196
6ac0d63b297f Optimized searching a bit for cases where we can restrict search range by
Timo Sirainen <tss@iki.fi>
parents: 4062
diff changeset
1891 }