annotate src/doveadm/dsync/dsync-mailbox.c @ 22715:20415dd0b85a

dsync: Add per-mailbox sync lock that is always used. Both importing and exporting gets the lock before they even sync the mailbox. The lock is kept until the import/export finishes. This guarantees that no matter how dsync is run, two dsyncs can't be working on the same mailbox at the same time. This lock is in addition to the optional per-user lock enabled by the -l parameter. If the -l parameter is used, the same lock timeout is used for the per-mailbox lock. Otherwise 30s timeout is used. This should help to avoid email duplication when replication is enabled for public namespaces, and maybe in some other rare situations as well.
author Timo Sirainen <timo.sirainen@dovecot.fi>
date Thu, 28 Dec 2017 14:10:23 +0200
parents cb108f786fb4
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
22713
cb108f786fb4 Updated copyright notices to include the year 2018.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 21390
diff changeset
1 /* Copyright (c) 2013-2018 Dovecot authors, see the included COPYING file */
16025
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
2
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
3 #include "lib.h"
16054
a4c01707feb7 Compiler warning fix.
Timo Sirainen <tss@iki.fi>
parents: 16052
diff changeset
4 #include "istream.h"
22715
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
5 #include "mail-storage-private.h"
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
6 #include "dsync-brain-private.h"
16025
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
7 #include "dsync-mailbox.h"
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
8
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
9 void dsync_mailbox_attribute_dup(pool_t pool,
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
10 const struct dsync_mailbox_attribute *src,
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
11 struct dsync_mailbox_attribute *dest_r)
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
12 {
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
13 dest_r->type = src->type;
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
14 dest_r->key = p_strdup(pool, src->key);
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
15 dest_r->value = p_strdup(pool, src->value);
16052
0e5a359b7b7f lib-storage: Mailbox attributes can now be accessed via istreams.
Timo Sirainen <tss@iki.fi>
parents: 16025
diff changeset
16 if (src->value_stream != NULL) {
0e5a359b7b7f lib-storage: Mailbox attributes can now be accessed via istreams.
Timo Sirainen <tss@iki.fi>
parents: 16025
diff changeset
17 dest_r->value_stream = src->value_stream;
0e5a359b7b7f lib-storage: Mailbox attributes can now be accessed via istreams.
Timo Sirainen <tss@iki.fi>
parents: 16025
diff changeset
18 i_stream_ref(dest_r->value_stream);
0e5a359b7b7f lib-storage: Mailbox attributes can now be accessed via istreams.
Timo Sirainen <tss@iki.fi>
parents: 16025
diff changeset
19 }
16025
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
20
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
21 dest_r->deleted = src->deleted;
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
22 dest_r->last_change = src->last_change;
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
23 dest_r->modseq = src->modseq;
c6082de4bf5b dsync: Added support for syncing mailbox attributes.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
24 }
22715
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
25
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
26 int dsync_mailbox_lock(struct dsync_brain *brain, struct mailbox *box,
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
27 struct file_lock **lock_r)
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
28 {
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
29 const char *path, *error;
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
30 int ret;
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
31
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
32 /* Make sure the mailbox is open - locking requires it */
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
33 if (mailbox_open(box) < 0) {
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
34 i_error("Can't open mailbox %s: %s", mailbox_get_vname(box),
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
35 mailbox_get_last_internal_error(box, &brain->mail_error));
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
36 return -1;
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
37 }
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
38
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
39 ret = mailbox_get_path_to(box, MAILBOX_LIST_PATH_TYPE_INDEX, &path);
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
40 if (ret < 0) {
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
41 i_error("Can't get mailbox %s path: %s", mailbox_get_vname(box),
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
42 mailbox_get_last_internal_error(box, &brain->mail_error));
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
43 return -1;
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
44 }
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
45 if (ret == 0) {
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
46 /* No index files - don't do any locking. In theory we still
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
47 could, but this lock is mainly meant to prevent replication
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
48 problems, and replication wouldn't work without indexes. */
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
49 *lock_r = NULL;
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
50 return 0;
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
51 }
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
52
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
53 if (mailbox_lock_file_create(box, DSYNC_MAILBOX_LOCK_FILENAME,
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
54 brain->mailbox_lock_timeout_secs,
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
55 lock_r, &error) <= 0) {
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
56 i_error("Failed to lock mailbox %s for dsyncing: %s",
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
57 box->vname, error);
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
58 return -1;
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
59 }
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
60 return 0;
20415dd0b85a dsync: Add per-mailbox sync lock that is always used.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 22713
diff changeset
61 }