dovecot/core-2.2: src/doveadm/dsync/dsync-brain.c annotate

annotate src/doveadm/dsync/dsync-brain.c @ 21256:4ac3461df334

dsync: When logging "Mailbox changed caused a desync", log also the reason. The reason is usually somewhere in the debug logs, but it's difficult to find from there.

author	Timo Sirainen <timo.sirainen@dovecot.fi>
date	Thu, 01 Dec 2016 19:06:10 +0200
parents	0c4e5c2725a3
children	9316919c5932

rev	line source
19552 0f22db71df7a global: freshen copyright Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19318 diff changeset	1 /* Copyright (c) 2013-2016 Dovecot authors, see the included COPYING file */
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	4 #include "array.h"
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "hash.h"
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	6 #include "hostpid.h"
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	7 #include "str.h"
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	8 #include "process-title.h"
16540 84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	9 #include "settings-parser.h"
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	10 #include "master-service.h"
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	11 #include "master-service-settings.h"
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	12 #include "mail-namespace.h"
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	13 #include "dsync-mailbox-tree.h"
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	14 #include "dsync-ibc.h"
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	15 #include "dsync-brain-private.h"
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	16 #include "dsync-mailbox-import.h"
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	17 #include "dsync-mailbox-export.h"
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	18
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	19 #include <sys/stat.h>
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	20
16538 83bea7e1f136 dsync: State names were wrong in debug/error messages. Timo Sirainen <tss@iki.fi> parents: 16503 diff changeset	21 static const char *dsync_state_names[] = {
83bea7e1f136 dsync: State names were wrong in debug/error messages. Timo Sirainen <tss@iki.fi> parents: 16503 diff changeset	22 "master_recv_handshake",
83bea7e1f136 dsync: State names were wrong in debug/error messages. Timo Sirainen <tss@iki.fi> parents: 16503 diff changeset	23 "slave_recv_handshake",
83bea7e1f136 dsync: State names were wrong in debug/error messages. Timo Sirainen <tss@iki.fi> parents: 16503 diff changeset	24 "master_send_last_common",
83bea7e1f136 dsync: State names were wrong in debug/error messages. Timo Sirainen <tss@iki.fi> parents: 16503 diff changeset	25 "slave_recv_last_common",
15464 af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	26 "send_mailbox_tree",
af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	27 "send_mailbox_tree_deletes",
af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	28 "recv_mailbox_tree",
af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	29 "recv_mailbox_tree_deletes",
af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	30 "master_send_mailbox",
af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	31 "slave_recv_mailbox",
af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	32 "sync_mails",
18351 b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	33 "finish",
15464 af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	34 "done"
af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	35 };
af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	36
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	37 static void dsync_brain_mailbox_states_dump(struct dsync_brain *brain);
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	38
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	39 static const char dsync_brain_get_proctitle(struct dsync_brain brain)
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	40 {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	41 string_t *str = t_str_new(128);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	42 const char import_title, export_title;
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	43
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	44 str_append_c(str, '[');
17279 fac4e7320676 dsync: If dsync was started via doveadm-server, show the remote client's IP address in ps output. Timo Sirainen <tss@iki.fi> parents: 17271 diff changeset	45 if (brain->process_title_prefix != NULL)
fac4e7320676 dsync: If dsync was started via doveadm-server, show the remote client's IP address in ps output. Timo Sirainen <tss@iki.fi> parents: 17271 diff changeset	46 str_append(str, brain->process_title_prefix);
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	47 str_append(str, brain->user->username);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	48 if (brain->box == NULL) {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	49 str_append_c(str, ' ');
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	50 str_append(str, dsync_state_names[brain->state]);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	51 } else {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	52 str_append_c(str, ' ');
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	53 str_append(str, mailbox_get_vname(brain->box));
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	54 import_title = brain->box_importer == NULL ? "" :
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	55 dsync_mailbox_import_get_proctitle(brain->box_importer);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	56 export_title = brain->box_exporter == NULL ? "" :
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	57 dsync_mailbox_export_get_proctitle(brain->box_exporter);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	58 if (import_title[0] == '\0' && export_title[0] == '\0') {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	59 str_printfa(str, " send:%s recv:%s",
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	60 dsync_box_state_names[brain->box_send_state],
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	61 dsync_box_state_names[brain->box_recv_state]);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	62 } else {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	63 if (import_title[0] != '\0') {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	64 str_append(str, " import:");
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	65 str_append(str, import_title);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	66 }
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	67 if (export_title[0] != '\0') {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	68 str_append(str, " export:");
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	69 str_append(str, export_title);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	70 }
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	71 }
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	72 }
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	73 str_append_c(str, ']');
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	74 return str_c(str);
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	75 }
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	76
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	77 static void dsync_brain_run_io(void *context)
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	78 {
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	79 struct dsync_brain *brain = context;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	80 bool changed, try_pending;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	81
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	82 if (dsync_ibc_has_failed(brain->ibc)) {
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	83 io_loop_stop(current_ioloop);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	84 brain->failed = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	85 return;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	86 }
11687 b37c85676f8e dsync: When -v parameter is given, show progress counter of saving new messages. Timo Sirainen <tss@iki.fi> parents: 11684 diff changeset	87
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	88 try_pending = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	89 do {
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	90 if (!dsync_brain_run(brain, &changed)) {
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	91 io_loop_stop(current_ioloop);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	92 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	93 }
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	94 if (changed)
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	95 try_pending = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	96 else if (try_pending) {
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	97 if (dsync_ibc_has_pending_data(brain->ibc))
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	98 changed = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	99 try_pending = FALSE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	100 }
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	101 } while (changed);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	102 }
11727 42dfcf9c896b dsync backup: Fail if it looks like backup is running in wrong direction. Timo Sirainen <tss@iki.fi> parents: 11688 diff changeset	103
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	104 static struct dsync_brain *
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	105 dsync_brain_common_init(struct mail_user user, struct dsync_ibc ibc)
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	106 {
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	107 struct dsync_brain *brain;
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	108 const struct master_service_settings *service_set;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	109 pool_t pool;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	110
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	111 service_set = master_service_settings_get(master_service);
16801 56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	112 mail_user_ref(user);
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	113
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	114 pool = pool_alloconly_create("dsync brain", 10240);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	115 brain = p_new(pool, struct dsync_brain, 1);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	116 brain->pool = pool;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	117 brain->user = user;
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	118 brain->ibc = ibc;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	119 brain->sync_type = DSYNC_BRAIN_SYNC_TYPE_UNKNOWN;
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	120 brain->lock_fd = -1;
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	121 brain->verbose_proctitle = service_set->verbose_proctitle;
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	122 hash_table_create(&brain->mailbox_states, pool, 0,
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	123 guid_128_hash, guid_128_cmp);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	124 p_array_init(&brain->remote_mailbox_states, pool, 64);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	125 return brain;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	126 }
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	127
15231 454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	128 static void
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	129 dsync_brain_set_flags(struct dsync_brain *brain, enum dsync_brain_flags flags)
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	130 {
15744 b4e2b3b54f0a dsync: Renamed "guid_requests" to "mail_requests" Timo Sirainen <tss@iki.fi> parents: 15715 diff changeset	131 brain->mail_requests =
b4e2b3b54f0a dsync: Renamed "guid_requests" to "mail_requests" Timo Sirainen <tss@iki.fi> parents: 15715 diff changeset	132 (flags & DSYNC_BRAIN_FLAG_SEND_MAIL_REQUESTS) != 0;
15231 454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	133 brain->backup_send = (flags & DSYNC_BRAIN_FLAG_BACKUP_SEND) != 0;
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	134 brain->backup_recv = (flags & DSYNC_BRAIN_FLAG_BACKUP_RECV) != 0;
15464 af3b86a16ac3 dsync: Added debugging and rawlogging support. Timo Sirainen <tss@iki.fi> parents: 15355 diff changeset	135 brain->debug = (flags & DSYNC_BRAIN_FLAG_DEBUG) != 0;
15836 607758d425e7 dsync: Renamed -a parameter to -N. It also now skips invisible namespaces. Timo Sirainen <tss@iki.fi> parents: 15825 diff changeset	136 brain->sync_visible_namespaces =
607758d425e7 dsync: Renamed -a parameter to -N. It also now skips invisible namespaces. Timo Sirainen <tss@iki.fi> parents: 15825 diff changeset	137 (flags & DSYNC_BRAIN_FLAG_SYNC_VISIBLE_NAMESPACES) != 0;
16084 38774adaddaf dsync: -m '' parameter now syncs mailbox list, but no actual mails. Timo Sirainen <tss@iki.fi> parents: 15986 diff changeset	138 brain->no_mail_sync = (flags & DSYNC_BRAIN_FLAG_NO_MAIL_SYNC) != 0;
16561 0144704e1b99 dsync: Added -1 parameter to do a "one way sync" without reverting changes. Timo Sirainen <tss@iki.fi> parents: 16541 diff changeset	139 brain->no_backup_overwrite =
0144704e1b99 dsync: Added -1 parameter to do a "one way sync" without reverting changes. Timo Sirainen <tss@iki.fi> parents: 16541 diff changeset	140 (flags & DSYNC_BRAIN_FLAG_NO_BACKUP_OVERWRITE) != 0;
17845 d2e1b3f6d13b dsync: Added DSYNC_BRAIN_FLAG_NO_MAIL_PREFETCH to avoid opening mails unnecessarily. Timo Sirainen <tss@iki.fi> parents: 17800 diff changeset	141 brain->no_mail_prefetch =
d2e1b3f6d13b dsync: Added DSYNC_BRAIN_FLAG_NO_MAIL_PREFETCH to avoid opening mails unnecessarily. Timo Sirainen <tss@iki.fi> parents: 17800 diff changeset	142 (flags & DSYNC_BRAIN_FLAG_NO_MAIL_PREFETCH) != 0;
19021 3fc9658c9712 dsync: Added -D parameter to disable mailbox renaming. Timo Sirainen <tss@iki.fi> parents: 18371 diff changeset	143 brain->no_mailbox_renames =
3fc9658c9712 dsync: Added -D parameter to disable mailbox renaming. Timo Sirainen <tss@iki.fi> parents: 18371 diff changeset	144 (flags & DSYNC_BRAIN_FLAG_NO_MAILBOX_RENAMES) != 0;
19318 f9a143c630a5 dsync: Added DSYNC_BRAIN_FLAG_NO_NOTIFY to enable MAILBOX_TRANSACTION_FLAG_NO_NOTIFY Timo Sirainen <tss@iki.fi> parents: 19136 diff changeset	145 brain->no_notify = (flags & DSYNC_BRAIN_FLAG_NO_NOTIFY) != 0;
20595 40ce04c672a4 dsync: Add support for features Aki Tuomi <aki.tuomi@dovecot.fi> parents: 20458 diff changeset	146 brain->empty_hdr_workaround = (flags & DSYNC_BRAIN_FLAG_EMPTY_HDR_WORKAROUND) != 0;
15231 454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	147 }
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	148
18180 39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	149 static void
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	150 dsync_brain_open_virtual_all_box(struct dsync_brain *brain,
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	151 const char *vname)
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	152 {
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	153 struct mail_namespace *ns;
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	154
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	155 ns = mail_namespace_find(brain->user->namespaces, vname);
18287 06c4c42549a7 dsync: Open mailboxes with readonly-flag whenever possible. Timo Sirainen <tss@iki.fi> parents: 18181 diff changeset	156 brain->virtual_all_box =
06c4c42549a7 dsync: Open mailboxes with readonly-flag whenever possible. Timo Sirainen <tss@iki.fi> parents: 18181 diff changeset	157 mailbox_alloc(ns->list, vname, MAILBOX_FLAG_READONLY);
18180 39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	158 }
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	159
9736 d9a96da46d4a dsync: Lots of updates and fixes. Timo Sirainen <tss@iki.fi> parents: 9686 diff changeset	160 struct dsync_brain *
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	161 dsync_brain_master_init(struct mail_user user, struct dsync_ibc ibc,
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	162 enum dsync_brain_sync_type sync_type,
16396 2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	163 enum dsync_brain_flags flags,
2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	164 const struct dsync_brain_settings *set)
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	165 {
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	166 struct dsync_ibc_settings ibc_set;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	167 struct dsync_brain *brain;
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	168 struct mail_namespace const nsp;
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	169 string_t *sync_ns_str = NULL;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	170 const char *error;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	171
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	172 i_assert(sync_type != DSYNC_BRAIN_SYNC_TYPE_UNKNOWN);
16396 2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	173 i_assert(sync_type != DSYNC_BRAIN_SYNC_TYPE_STATE \|\|
2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	174 (set->state != NULL && *set->state != '\0'));
16538 83bea7e1f136 dsync: State names were wrong in debug/error messages. Timo Sirainen <tss@iki.fi> parents: 16503 diff changeset	175 i_assert(N_ELEMENTS(dsync_state_names) == DSYNC_STATE_DONE+1);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	176
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	177 brain = dsync_brain_common_init(user, ibc);
17279 fac4e7320676 dsync: If dsync was started via doveadm-server, show the remote client's IP address in ps output. Timo Sirainen <tss@iki.fi> parents: 17271 diff changeset	178 brain->process_title_prefix =
fac4e7320676 dsync: If dsync was started via doveadm-server, show the remote client's IP address in ps output. Timo Sirainen <tss@iki.fi> parents: 17271 diff changeset	179 p_strdup(brain->pool, set->process_title_prefix);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	180 brain->sync_type = sync_type;
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	181 if (array_count(&set->sync_namespaces) > 0) {
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	182 sync_ns_str = t_str_new(128);
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	183 p_array_init(&brain->sync_namespaces, brain->pool,
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	184 array_count(&set->sync_namespaces));
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	185 array_foreach(&set->sync_namespaces, nsp) {
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	186 str_append(sync_ns_str, (*nsp)->prefix);
17219 f0b8cf119b7f dsync: Fixed giving -n "" followed by other -n parameters. Timo Sirainen <tss@iki.fi> parents: 17200 diff changeset	187 str_append_c(sync_ns_str, '\n');
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	188 array_append(&brain->sync_namespaces, nsp, 1);
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	189 }
17221 e75851b65900 dsync: Fix to previous commit to actually compile.. Timo Sirainen <tss@iki.fi> parents: 17219 diff changeset	190 str_delete(sync_ns_str, str_len(sync_ns_str)-1, 1);
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	191 }
17800 eff79a80e0c9 dsync: Moved all doveadm-specific code to doveadm-dsync.c Timo Sirainen <tss@iki.fi> parents: 17377 diff changeset	192 brain->alt_char = set->mailbox_alt_char == '\0' ? '_' :
eff79a80e0c9 dsync: Moved all doveadm-specific code to doveadm-dsync.c Timo Sirainen <tss@iki.fi> parents: 17377 diff changeset	193 set->mailbox_alt_char;
18177 f393f63764e0 dsync: Added -t <timestamp> parameter to save only mails newer than <timestamp> Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	194 brain->sync_since_timestamp = set->sync_since_timestamp;
20831 0c4e5c2725a3 doveadm-sync: Add end-date support Aki Tuomi <aki.tuomi@dovecot.fi> parents: 20633 diff changeset	195 brain->sync_until_timestamp = set->sync_until_timestamp;
20633 d54651ba988a doveadm sync/backup: Added -S <max size> parameter to skip too large mails. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20595 diff changeset	196 brain->sync_max_size = set->sync_max_size;
18181 35e4a6ae8d85 dsync: Added -F parameter to sync only mails with[out] specific flag. Timo Sirainen <tss@iki.fi> parents: 18180 diff changeset	197 brain->sync_flag = p_strdup(brain->pool, set->sync_flag);
16396 2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	198 brain->sync_box = p_strdup(brain->pool, set->sync_box);
16398 4883a8e1db13 dsync: Added -x parameter to exclude mailboxes from sync. Timo Sirainen <tss@iki.fi> parents: 16396 diff changeset	199 brain->exclude_mailboxes = set->exclude_mailboxes == NULL ? NULL :
4883a8e1db13 dsync: Added -x parameter to exclude mailboxes from sync. Timo Sirainen <tss@iki.fi> parents: 16396 diff changeset	200 p_strarray_dup(brain->pool, set->exclude_mailboxes);
16396 2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	201 memcpy(brain->sync_box_guid, set->sync_box_guid,
2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	202 sizeof(brain->sync_box_guid));
2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	203 brain->lock_timeout = set->lock_timeout_secs;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	204 brain->master_brain = TRUE;
15231 454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	205 dsync_brain_set_flags(brain, flags);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	206
18180 39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	207 if (set->virtual_all_box != NULL)
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	208 dsync_brain_open_virtual_all_box(brain, set->virtual_all_box);
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	209
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	210 if (sync_type != DSYNC_BRAIN_SYNC_TYPE_STATE)
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	211 ;
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	212 else if (dsync_mailbox_states_import(brain->mailbox_states, brain->pool,
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	213 set->state, &error) < 0) {
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	214 hash_table_clear(brain->mailbox_states, FALSE);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	215 i_error("Saved sync state is invalid, "
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	216 "falling back to full sync: %s", error);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	217 brain->sync_type = sync_type = DSYNC_BRAIN_SYNC_TYPE_FULL;
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	218 } else {
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	219 if (brain->debug) {
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	220 i_debug("brain %c: Imported mailbox states:",
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	221 brain->master_brain ? 'M' : 'S');
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	222 dsync_brain_mailbox_states_dump(brain);
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	223 }
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	224 }
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	225 dsync_brain_mailbox_trees_init(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	226
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	227 memset(&ibc_set, 0, sizeof(ibc_set));
15815 e7aabd79c9d5 dsync: Use full hostname+domain when comparing if hosts are different in locking. Timo Sirainen <tss@iki.fi> parents: 15782 diff changeset	228 ibc_set.hostname = my_hostdomain();
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	229 ibc_set.sync_ns_prefixes = sync_ns_str == NULL ?
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	230 NULL : str_c(sync_ns_str);
16396 2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	231 ibc_set.sync_box = set->sync_box;
18180 39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	232 ibc_set.virtual_all_box = set->virtual_all_box;
16398 4883a8e1db13 dsync: Added -x parameter to exclude mailboxes from sync. Timo Sirainen <tss@iki.fi> parents: 16396 diff changeset	233 ibc_set.exclude_mailboxes = set->exclude_mailboxes;
18177 f393f63764e0 dsync: Added -t <timestamp> parameter to save only mails newer than <timestamp> Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	234 ibc_set.sync_since_timestamp = set->sync_since_timestamp;
20831 0c4e5c2725a3 doveadm-sync: Add end-date support Aki Tuomi <aki.tuomi@dovecot.fi> parents: 20633 diff changeset	235 ibc_set.sync_until_timestamp = set->sync_until_timestamp;
20633 d54651ba988a doveadm sync/backup: Added -S <max size> parameter to skip too large mails. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20595 diff changeset	236 ibc_set.sync_max_size = set->sync_max_size;
18181 35e4a6ae8d85 dsync: Added -F parameter to sync only mails with[out] specific flag. Timo Sirainen <tss@iki.fi> parents: 18180 diff changeset	237 ibc_set.sync_flags = set->sync_flag;
16396 2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	238 memcpy(ibc_set.sync_box_guid, set->sync_box_guid,
16110 c51873a8e0d9 doveadm sync/backup: Added -g <guid> to sync only the specified mailbox (by GUID) Timo Sirainen <tss@iki.fi> parents: 16089 diff changeset	239 sizeof(ibc_set.sync_box_guid));
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	240 ibc_set.sync_type = sync_type;
19646 25f06710e671 dsync: When comparing headers' hashes to match messages, try to normalize the input. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	241 ibc_set.hdr_hash_v2 = TRUE;
16396 2d6a3035a6f7 dsync: Small code cleanup. Timo Sirainen <tss@iki.fi> parents: 16253 diff changeset	242 ibc_set.lock_timeout = set->lock_timeout_secs;
15231 454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	243 /* reverse the backup direction for the slave */
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	244 ibc_set.brain_flags = flags & ~(DSYNC_BRAIN_FLAG_BACKUP_SEND \|
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	245 DSYNC_BRAIN_FLAG_BACKUP_RECV);
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	246 if ((flags & DSYNC_BRAIN_FLAG_BACKUP_SEND) != 0)
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	247 ibc_set.brain_flags \|= DSYNC_BRAIN_FLAG_BACKUP_RECV;
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	248 else if ((flags & DSYNC_BRAIN_FLAG_BACKUP_RECV) != 0)
454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	249 ibc_set.brain_flags \|= DSYNC_BRAIN_FLAG_BACKUP_SEND;
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	250 dsync_ibc_send_handshake(ibc, &ibc_set);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	251
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	252 dsync_ibc_set_io_callback(ibc, dsync_brain_run_io, brain);
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	253 brain->state = DSYNC_STATE_MASTER_RECV_HANDSHAKE;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	254 return brain;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	255 }
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	256
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	257 struct dsync_brain *
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	258 dsync_brain_slave_init(struct mail_user user, struct dsync_ibc ibc,
17279 fac4e7320676 dsync: If dsync was started via doveadm-server, show the remote client's IP address in ps output. Timo Sirainen <tss@iki.fi> parents: 17271 diff changeset	259 bool local, const char *process_title_prefix)
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	260 {
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	261 struct dsync_ibc_settings ibc_set;
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	262 struct dsync_brain *brain;
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	263
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	264 brain = dsync_brain_common_init(user, ibc);
17279 fac4e7320676 dsync: If dsync was started via doveadm-server, show the remote client's IP address in ps output. Timo Sirainen <tss@iki.fi> parents: 17271 diff changeset	265 brain->process_title_prefix =
fac4e7320676 dsync: If dsync was started via doveadm-server, show the remote client's IP address in ps output. Timo Sirainen <tss@iki.fi> parents: 17271 diff changeset	266 p_strdup(brain->pool, process_title_prefix);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	267 brain->state = DSYNC_STATE_SLAVE_RECV_HANDSHAKE;
11734 ba9f33f9c6c3 dsync: If verbosity is enabled, log why desyncing was caused. Timo Sirainen <tss@iki.fi> parents: 11727 diff changeset	268
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	269 if (local) {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	270 /* both master and slave are running within the same process,
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	271 update the proctitle only for master. */
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	272 brain->verbose_proctitle = FALSE;
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	273 }
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	274
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	275 memset(&ibc_set, 0, sizeof(ibc_set));
19646 25f06710e671 dsync: When comparing headers' hashes to match messages, try to normalize the input. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	276 ibc_set.hdr_hash_v2 = TRUE;
15815 e7aabd79c9d5 dsync: Use full hostname+domain when comparing if hosts are different in locking. Timo Sirainen <tss@iki.fi> parents: 15782 diff changeset	277 ibc_set.hostname = my_hostdomain();
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	278 dsync_ibc_send_handshake(ibc, &ibc_set);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	279
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	280 dsync_ibc_set_io_callback(ibc, dsync_brain_run_io, brain);
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	281 return brain;
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	282 }
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	283
16801 56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	284 static void dsync_brain_purge(struct dsync_brain *brain)
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	285 {
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	286 struct mail_namespace *ns;
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	287 struct mail_storage *storage;
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	288
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	289 for (ns = brain->user->namespaces; ns != NULL; ns = ns->next) {
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	290 if (!dsync_brain_want_namespace(brain, ns))
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	291 continue;
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	292
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	293 storage = mail_namespace_get_default_storage(ns);
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	294 if (mail_storage_purge(storage) < 0) {
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	295 i_error("Purging namespace '%s' failed: %s", ns->prefix,
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	296 mail_storage_get_last_error(storage, NULL));
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	297 }
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	298 }
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	299 }
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	300
18371 b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	301 int dsync_brain_deinit(struct dsync_brain *_brain, enum mail_error error_r)
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	302 {
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	303 struct dsync_brain brain = _brain;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	304 int ret;
11680 49b226835cd2 dsync: Another try at not increasing mailbox uidnext/highestmodseq on failure Timo Sirainen <tss@iki.fi> parents: 11677 diff changeset	305
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	306 *_brain = NULL;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	307
15986 f6377e089dee dsync: If I/O gets stalled, log the state in which it happened. Timo Sirainen <tss@iki.fi> parents: 15836 diff changeset	308 if (dsync_ibc_has_timed_out(brain->ibc)) {
f6377e089dee dsync: If I/O gets stalled, log the state in which it happened. Timo Sirainen <tss@iki.fi> parents: 15836 diff changeset	309 i_error("Timeout during state=%s%s",
f6377e089dee dsync: If I/O gets stalled, log the state in which it happened. Timo Sirainen <tss@iki.fi> parents: 15836 diff changeset	310 dsync_state_names[brain->state],
f6377e089dee dsync: If I/O gets stalled, log the state in which it happened. Timo Sirainen <tss@iki.fi> parents: 15836 diff changeset	311 brain->state != DSYNC_STATE_SYNC_MAILS ? "" :
f6377e089dee dsync: If I/O gets stalled, log the state in which it happened. Timo Sirainen <tss@iki.fi> parents: 15836 diff changeset	312 t_strdup_printf(" (send=%s recv=%s)",
f6377e089dee dsync: If I/O gets stalled, log the state in which it happened. Timo Sirainen <tss@iki.fi> parents: 15836 diff changeset	313 dsync_box_state_names[brain->box_send_state],
f6377e089dee dsync: If I/O gets stalled, log the state in which it happened. Timo Sirainen <tss@iki.fi> parents: 15836 diff changeset	314 dsync_box_state_names[brain->box_recv_state]));
f6377e089dee dsync: If I/O gets stalled, log the state in which it happened. Timo Sirainen <tss@iki.fi> parents: 15836 diff changeset	315 }
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	316 if (dsync_ibc_has_failed(brain->ibc) \|\|
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	317 brain->state != DSYNC_STATE_DONE)
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	318 brain->failed = TRUE;
15632 40dd9a080b6e dsync: Close mail streams earlier on failures to avoid assert-crashing Timo Sirainen <tss@iki.fi> parents: 15594 diff changeset	319 dsync_ibc_close_mail_streams(brain->ibc);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	320
16801 56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	321 if (brain->purge && !brain->failed)
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	322 dsync_brain_purge(brain);
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	323
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	324 if (brain->box != NULL)
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	325 dsync_brain_sync_mailbox_deinit(brain);
18180 39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	326 if (brain->virtual_all_box != NULL)
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	327 mailbox_free(&brain->virtual_all_box);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	328 if (brain->local_tree_iter != NULL)
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	329 dsync_mailbox_tree_iter_deinit(&brain->local_tree_iter);
16541 07e314199f56 dsync: Don't crash at deinit when dsync fails early. Timo Sirainen <tss@iki.fi> parents: 16540 diff changeset	330 if (brain->local_mailbox_tree != NULL)
07e314199f56 dsync: Don't crash at deinit when dsync fails early. Timo Sirainen <tss@iki.fi> parents: 16540 diff changeset	331 dsync_mailbox_tree_deinit(&brain->local_mailbox_tree);
07e314199f56 dsync: Don't crash at deinit when dsync fails early. Timo Sirainen <tss@iki.fi> parents: 16540 diff changeset	332 if (brain->remote_mailbox_tree != NULL)
07e314199f56 dsync: Don't crash at deinit when dsync fails early. Timo Sirainen <tss@iki.fi> parents: 16540 diff changeset	333 dsync_mailbox_tree_deinit(&brain->remote_mailbox_tree);
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	334 if (brain->mailbox_states_iter != NULL)
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	335 hash_table_iterate_deinit(&brain->mailbox_states_iter);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	336 hash_table_destroy(&brain->mailbox_states);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	337
17377 a21ac1261487 dsync: mailbox_metadata.cache_fields must be copied to permanent memory. Timo Sirainen <tss@iki.fi> parents: 17282 diff changeset	338 if (brain->dsync_box_pool != NULL)
a21ac1261487 dsync: mailbox_metadata.cache_fields must be copied to permanent memory. Timo Sirainen <tss@iki.fi> parents: 17282 diff changeset	339 pool_unref(&brain->dsync_box_pool);
a21ac1261487 dsync: mailbox_metadata.cache_fields must be copied to permanent memory. Timo Sirainen <tss@iki.fi> parents: 17282 diff changeset	340
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	341 if (brain->lock_fd != -1) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	342 /* unlink the lock file before it gets unlocked */
19136 fefaa6d09a81 Replaced unlink() calls with i_unlink() wherever possible. Timo Sirainen <tss@iki.fi>* parents: 19021 diff changeset	343 i_unlink(brain->lock_path);
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	344 file_lock_free(&brain->lock);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	345 i_close_fd(&brain->lock_fd);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	346 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	347
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	348 ret = brain->failed ? -1 : 0;
16801 56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	349 mail_user_unref(&brain->user);
18371 b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	350
b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	351 *error_r = !brain->failed ? 0 :
b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	352 (brain->mail_error == 0 ? MAIL_ERROR_TEMP : brain->mail_error);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	353 pool_unref(&brain->pool);
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	354 return ret;
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	355 }
34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	356
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	357 static int
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	358 dsync_brain_lock(struct dsync_brain brain, const char remote_hostname)
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	359 {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	360 struct stat st1, st2;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	361 const char *home;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	362 int ret;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	363
15825 df29194c1ab6 dsync: Fix to checking which side should do the locking. Timo Sirainen <tss@iki.fi> parents: 15816 diff changeset	364 if ((ret = strcmp(remote_hostname, my_hostdomain())) < 0) {
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	365 /* locking done by remote */
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	366 return 0;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	367 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	368 if (ret == 0 && !brain->master_brain) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	369 /* running dsync within the same server.
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	370 locking done by master brain. */
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	371 return 0;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	372 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	373
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	374 if ((ret = mail_user_get_home(brain->user, &home)) < 0) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	375 i_error("Couldn't look up user's home dir");
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	376 return -1;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	377 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	378 if (ret == 0) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	379 i_error("User has no home directory");
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	380 return -1;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	381 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	382
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	383 brain->lock_path = p_strconcat(brain->pool, home,
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	384 "/"DSYNC_LOCK_FILENAME, NULL);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	385 for (;;) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	386 brain->lock_fd = creat(brain->lock_path, 0600);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	387 if (brain->lock_fd == -1) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	388 i_error("Couldn't create lock %s: %m",
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	389 brain->lock_path);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	390 return -1;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	391 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	392
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	393 if (file_wait_lock(brain->lock_fd, brain->lock_path, F_WRLCK,
16089 b436c1f6bd06 dsync: Use fcntl() locking instead of flock(). Timo Sirainen <tss@iki.fi> parents: 16084 diff changeset	394 FILE_LOCK_METHOD_FCNTL, brain->lock_timeout,
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	395 &brain->lock) <= 0) {
16251 e41a13ae504d dsync: If dsync fails due to lock timeout, give a better error message. Timo Sirainen <tss@iki.fi> parents: 16222 diff changeset	396 if (errno == EAGAIN) {
e41a13ae504d dsync: If dsync fails due to lock timeout, give a better error message. Timo Sirainen <tss@iki.fi> parents: 16222 diff changeset	397 i_error("Couldn't lock %s: Timed out after %u seconds",
e41a13ae504d dsync: If dsync fails due to lock timeout, give a better error message. Timo Sirainen <tss@iki.fi> parents: 16222 diff changeset	398 brain->lock_path, brain->lock_timeout);
e41a13ae504d dsync: If dsync fails due to lock timeout, give a better error message. Timo Sirainen <tss@iki.fi> parents: 16222 diff changeset	399 } else {
e41a13ae504d dsync: If dsync fails due to lock timeout, give a better error message. Timo Sirainen <tss@iki.fi> parents: 16222 diff changeset	400 i_error("Couldn't lock %s: %m", brain->lock_path);
e41a13ae504d dsync: If dsync fails due to lock timeout, give a better error message. Timo Sirainen <tss@iki.fi> parents: 16222 diff changeset	401 }
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	402 break;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	403 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	404 if (fstat(brain->lock_fd, &st1) < 0) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	405 if (errno != ESTALE) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	406 i_error("fstat(%s) failed: %m", brain->lock_path);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	407 break;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	408 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	409 } else if (stat(brain->lock_path, &st2) < 0) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	410 if (errno != ENOENT) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	411 i_error("stat(%s) failed: %m", brain->lock_path);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	412 break;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	413 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	414 } else if (st1.st_ino == st2.st_ino) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	415 /* success */
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	416 return 0;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	417 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	418 /* file was recreated, try again */
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	419 i_close_fd(&brain->lock_fd);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	420 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	421 i_close_fd(&brain->lock_fd);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	422 return -1;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	423 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	424
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	425 static bool dsync_brain_master_recv_handshake(struct dsync_brain *brain)
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	426 {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	427 const struct dsync_ibc_settings *ibc_set;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	428
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	429 i_assert(brain->master_brain);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	430
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	431 if (dsync_ibc_recv_handshake(brain->ibc, &ibc_set) == 0)
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	432 return FALSE;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	433
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	434 if (brain->lock_timeout > 0) {
15816 adb9857a3879 dsync: If locking fails, fail instead of continuing. Timo Sirainen <tss@iki.fi> parents: 15815 diff changeset	435 if (dsync_brain_lock(brain, ibc_set->hostname) < 0) {
adb9857a3879 dsync: If locking fails, fail instead of continuing. Timo Sirainen <tss@iki.fi> parents: 15815 diff changeset	436 brain->failed = TRUE;
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	437 return FALSE;
15816 adb9857a3879 dsync: If locking fails, fail instead of continuing. Timo Sirainen <tss@iki.fi> parents: 15815 diff changeset	438 }
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	439 }
19646 25f06710e671 dsync: When comparing headers' hashes to match messages, try to normalize the input. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	440 brain->hdr_hash_v2 = ibc_set->hdr_hash_v2;
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	441
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	442 brain->state = brain->sync_type == DSYNC_BRAIN_SYNC_TYPE_STATE ?
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	443 DSYNC_STATE_MASTER_SEND_LAST_COMMON :
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	444 DSYNC_STATE_SEND_MAILBOX_TREE;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	445 return TRUE;
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	446 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	447
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	448 static bool dsync_brain_slave_recv_handshake(struct dsync_brain *brain)
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	449 {
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	450 const struct dsync_ibc_settings *ibc_set;
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	451 struct mail_namespace *ns;
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	452 const char const prefixes;
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	453
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	454 i_assert(!brain->master_brain);
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	455
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	456 if (dsync_ibc_recv_handshake(brain->ibc, &ibc_set) == 0)
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	457 return FALSE;
19646 25f06710e671 dsync: When comparing headers' hashes to match messages, try to normalize the input. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	458 brain->hdr_hash_v2 = ibc_set->hdr_hash_v2;
10331 b5b253d35612 dsync: Fixed subscription syncing to work with namespace prefixes. Timo Sirainen <tss@iki.fi> parents: 10328 diff changeset	459
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	460 if (ibc_set->lock_timeout > 0) {
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	461 brain->lock_timeout = ibc_set->lock_timeout;
15816 adb9857a3879 dsync: If locking fails, fail instead of continuing. Timo Sirainen <tss@iki.fi> parents: 15815 diff changeset	462 if (dsync_brain_lock(brain, ibc_set->hostname) < 0) {
adb9857a3879 dsync: If locking fails, fail instead of continuing. Timo Sirainen <tss@iki.fi> parents: 15815 diff changeset	463 brain->failed = TRUE;
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	464 return FALSE;
15816 adb9857a3879 dsync: If locking fails, fail instead of continuing. Timo Sirainen <tss@iki.fi> parents: 15815 diff changeset	465 }
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	466 }
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	467
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	468 if (ibc_set->sync_ns_prefixes != NULL) {
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	469 p_array_init(&brain->sync_namespaces, brain->pool, 4);
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	470 prefixes = t_strsplit(ibc_set->sync_ns_prefixes, "\n");
17200 3e1a69e0cda9 dsync: Fixed using -n "" parameter Timo Sirainen <tss@iki.fi> parents: 17170 diff changeset	471 if (prefixes[0] == NULL) {
3e1a69e0cda9 dsync: Fixed using -n "" parameter Timo Sirainen <tss@iki.fi> parents: 17170 diff changeset	472 /* ugly workaround for strsplit API: there was one
3e1a69e0cda9 dsync: Fixed using -n "" parameter Timo Sirainen <tss@iki.fi> parents: 17170 diff changeset	473 prefix="" entry */
3e1a69e0cda9 dsync: Fixed using -n "" parameter Timo Sirainen <tss@iki.fi> parents: 17170 diff changeset	474 static const char *empty_prefix[] = { "", NULL };
3e1a69e0cda9 dsync: Fixed using -n "" parameter Timo Sirainen <tss@iki.fi> parents: 17170 diff changeset	475 prefixes = empty_prefix;
3e1a69e0cda9 dsync: Fixed using -n "" parameter Timo Sirainen <tss@iki.fi> parents: 17170 diff changeset	476 }
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	477 for (; *prefixes != NULL; prefixes++) {
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	478 ns = mail_namespace_find(brain->user->namespaces,
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	479 *prefixes);
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	480 if (ns == NULL) {
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	481 i_error("Namespace not found: '%s'", *prefixes);
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	482 brain->failed = TRUE;
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	483 return FALSE;
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	484 }
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	485 array_append(&brain->sync_namespaces, &ns, 1);
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	486 }
11676 cf7f6912af02 dsync: Added backup command, which syncs source to destination, discarding any changes in dest. Timo Sirainen <tss@iki.fi> parents: 11524 diff changeset	487 }
15471 88ac919f8afe dsync: Added back support for syncing only one mailbox (-m parameter) Timo Sirainen <tss@iki.fi> parents: 15469 diff changeset	488 brain->sync_box = p_strdup(brain->pool, ibc_set->sync_box);
16398 4883a8e1db13 dsync: Added -x parameter to exclude mailboxes from sync. Timo Sirainen <tss@iki.fi> parents: 16396 diff changeset	489 brain->exclude_mailboxes = ibc_set->exclude_mailboxes == NULL ? NULL :
4883a8e1db13 dsync: Added -x parameter to exclude mailboxes from sync. Timo Sirainen <tss@iki.fi> parents: 16396 diff changeset	490 p_strarray_dup(brain->pool, ibc_set->exclude_mailboxes);
18177 f393f63764e0 dsync: Added -t <timestamp> parameter to save only mails newer than <timestamp> Timo Sirainen <tss@iki.fi> parents: 18137 diff changeset	491 brain->sync_since_timestamp = ibc_set->sync_since_timestamp;
20831 0c4e5c2725a3 doveadm-sync: Add end-date support Aki Tuomi <aki.tuomi@dovecot.fi> parents: 20633 diff changeset	492 brain->sync_until_timestamp = ibc_set->sync_until_timestamp;
20633 d54651ba988a doveadm sync/backup: Added -S <max size> parameter to skip too large mails. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20595 diff changeset	493 brain->sync_max_size = ibc_set->sync_max_size;
18181 35e4a6ae8d85 dsync: Added -F parameter to sync only mails with[out] specific flag. Timo Sirainen <tss@iki.fi> parents: 18180 diff changeset	494 brain->sync_flag = p_strdup(brain->pool, ibc_set->sync_flags);
16110 c51873a8e0d9 doveadm sync/backup: Added -g <guid> to sync only the specified mailbox (by GUID) Timo Sirainen <tss@iki.fi> parents: 16089 diff changeset	495 memcpy(brain->sync_box_guid, ibc_set->sync_box_guid,
c51873a8e0d9 doveadm sync/backup: Added -g <guid> to sync only the specified mailbox (by GUID) Timo Sirainen <tss@iki.fi> parents: 16089 diff changeset	496 sizeof(brain->sync_box_guid));
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	497 i_assert(brain->sync_type == DSYNC_BRAIN_SYNC_TYPE_UNKNOWN);
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	498 brain->sync_type = ibc_set->sync_type;
16801 56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	499
15231 454d0563927d doveadm: "backup" command is working again. Timo Sirainen <tss@iki.fi> parents: 15037 diff changeset	500 dsync_brain_set_flags(brain, ibc_set->brain_flags);
16801 56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	501 /* this flag is only set on the remote slave brain */
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	502 brain->purge = (ibc_set->brain_flags &
56be613e8ece dsync: Added -P parameter to do a purge for the remote storage after syncing. Timo Sirainen <tss@iki.fi> parents: 16561 diff changeset	503 DSYNC_BRAIN_FLAG_PURGE_REMOTE) != 0;
11676 cf7f6912af02 dsync: Added backup command, which syncs source to destination, discarding any changes in dest. Timo Sirainen <tss@iki.fi> parents: 11524 diff changeset	504
18180 39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	505 if (ibc_set->virtual_all_box != NULL)
39d00448490f dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. Timo Sirainen <tss@iki.fi> parents: 18177 diff changeset	506 dsync_brain_open_virtual_all_box(brain, ibc_set->virtual_all_box);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	507 dsync_brain_mailbox_trees_init(brain);
11676 cf7f6912af02 dsync: Added backup command, which syncs source to destination, discarding any changes in dest. Timo Sirainen <tss@iki.fi> parents: 11524 diff changeset	508
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	509 if (brain->sync_type == DSYNC_BRAIN_SYNC_TYPE_STATE)
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	510 brain->state = DSYNC_STATE_SLAVE_RECV_LAST_COMMON;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	511 else
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	512 brain->state = DSYNC_STATE_SEND_MAILBOX_TREE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	513 return TRUE;
11727 42dfcf9c896b dsync backup: Fail if it looks like backup is running in wrong direction. Timo Sirainen <tss@iki.fi> parents: 11688 diff changeset	514 }
42dfcf9c896b dsync backup: Fail if it looks like backup is running in wrong direction. Timo Sirainen <tss@iki.fi> parents: 11688 diff changeset	515
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	516 static void dsync_brain_master_send_last_common(struct dsync_brain *brain)
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	517 {
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	518 struct dsync_mailbox_state *state;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	519 uint8_t *guid;
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	520 enum dsync_ibc_send_ret ret = DSYNC_IBC_SEND_RET_OK;
11727 42dfcf9c896b dsync backup: Fail if it looks like backup is running in wrong direction. Timo Sirainen <tss@iki.fi> parents: 11688 diff changeset	521
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	522 i_assert(brain->master_brain);
9736 d9a96da46d4a dsync: Lots of updates and fixes. Timo Sirainen <tss@iki.fi> parents: 9686 diff changeset	523
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	524 if (brain->mailbox_states_iter == NULL) {
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	525 brain->mailbox_states_iter =
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	526 hash_table_iterate_init(brain->mailbox_states);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	527 }
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	528
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	529 for (;;) {
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	530 if (ret == DSYNC_IBC_SEND_RET_FULL)
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	531 return;
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	532 if (!hash_table_iterate(brain->mailbox_states_iter,
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	533 brain->mailbox_states, &guid, &state))
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	534 break;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	535 ret = dsync_ibc_send_mailbox_state(brain->ibc, state);
9638 b11a3eda2477 dsync: Use expunged messages' GUIDs to determine what to do with missing messages at end of mailbox. Timo Sirainen <tss@iki.fi> parents: 9592 diff changeset	536 }
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	537 hash_table_iterate_deinit(&brain->mailbox_states_iter);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	538
16222 d79cf48f1072 dsync: Fixed talking to earlier dsync without mailbox attribute support. Timo Sirainen <tss@iki.fi> parents: 16110 diff changeset	539 dsync_ibc_send_end_of_list(brain->ibc, DSYNC_IBC_EOL_MAILBOX_STATE);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	540 brain->state = DSYNC_STATE_SEND_MAILBOX_TREE;
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	541 }
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	542
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	543 static void dsync_mailbox_state_add(struct dsync_brain *brain,
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	544 const struct dsync_mailbox_state *state)
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	545 {
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	546 struct dsync_mailbox_state *dupstate;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	547 uint8_t *guid_p;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	548
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	549 dupstate = p_new(brain->pool, struct dsync_mailbox_state, 1);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	550 dupstate = state;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	551 guid_p = dupstate->mailbox_guid;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	552 hash_table_insert(brain->mailbox_states, guid_p, dupstate);
11524 c955d4789553 dsync: Fixed syncing \noselect mailboxes. Timo Sirainen <tss@iki.fi> parents: 10976 diff changeset	553 }
c955d4789553 dsync: Fixed syncing \noselect mailboxes. Timo Sirainen <tss@iki.fi> parents: 10976 diff changeset	554
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	555 static bool dsync_brain_slave_recv_last_common(struct dsync_brain *brain)
10328 b63fd6156663 dsync: Added support for subscription syncing. Timo Sirainen <tss@iki.fi> parents: 10323 diff changeset	556 {
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	557 struct dsync_mailbox_state state;
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	558 enum dsync_ibc_recv_ret ret;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	559 bool changed = FALSE;
10328 b63fd6156663 dsync: Added support for subscription syncing. Timo Sirainen <tss@iki.fi> parents: 10323 diff changeset	560
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	561 i_assert(!brain->master_brain);
10328 b63fd6156663 dsync: Added support for subscription syncing. Timo Sirainen <tss@iki.fi> parents: 10323 diff changeset	562
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	563 while ((ret = dsync_ibc_recv_mailbox_state(brain->ibc, &state)) > 0) {
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	564 dsync_mailbox_state_add(brain, &state);
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	565 changed = TRUE;
10328 b63fd6156663 dsync: Added support for subscription syncing. Timo Sirainen <tss@iki.fi> parents: 10323 diff changeset	566 }
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	567 if (ret == DSYNC_IBC_RECV_RET_FINISHED) {
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	568 brain->state = DSYNC_STATE_SEND_MAILBOX_TREE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	569 changed = TRUE;
9736 d9a96da46d4a dsync: Lots of updates and fixes. Timo Sirainen <tss@iki.fi> parents: 9686 diff changeset	570 }
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	571 return changed;
10371 b99a19d5a93c dsync: Sync mailbox renames. Timo Sirainen <tss@iki.fi> parents: 10368 diff changeset	572 }
b99a19d5a93c dsync: Sync mailbox renames. Timo Sirainen <tss@iki.fi> parents: 10368 diff changeset	573
18351 b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	574 static bool dsync_brain_finish(struct dsync_brain *brain)
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	575 {
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	576 const char *error;
18371 b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	577 enum mail_error mail_error;
20458 16f9c0a46cbb dsync: When full resync is wanted in a stateful sync, output empty state. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20073 diff changeset	578 bool require_full_resync;
18371 b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	579 enum dsync_ibc_recv_ret ret;
18351 b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	580
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	581 if (!brain->master_brain) {
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	582 dsync_ibc_send_finish(brain->ibc,
18371 b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	583 brain->failed ? "dsync failed" : NULL,
20458 16f9c0a46cbb dsync: When full resync is wanted in a stateful sync, output empty state. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20073 diff changeset	584 brain->mail_error,
16f9c0a46cbb dsync: When full resync is wanted in a stateful sync, output empty state. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20073 diff changeset	585 brain->require_full_resync);
18351 b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	586 brain->state = DSYNC_STATE_DONE;
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	587 return TRUE;
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	588 }
20458 16f9c0a46cbb dsync: When full resync is wanted in a stateful sync, output empty state. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20073 diff changeset	589 ret = dsync_ibc_recv_finish(brain->ibc, &error, &mail_error,
16f9c0a46cbb dsync: When full resync is wanted in a stateful sync, output empty state. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20073 diff changeset	590 &require_full_resync);
18371 b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	591 if (ret == DSYNC_IBC_RECV_RET_TRYAGAIN)
18351 b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	592 return FALSE;
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	593 if (error != NULL) {
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	594 i_error("Remote dsync failed: %s", error);
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	595 brain->failed = TRUE;
18371 b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	596 if (mail_error != 0 &&
b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	597 (brain->mail_error == 0 \|\| brain->mail_error == MAIL_ERROR_TEMP))
b900b50085fc dsync: Use storage's mail_error to choose the doveadm exit code. Timo Sirainen <tss@iki.fi> parents: 18351 diff changeset	598 brain->mail_error = mail_error;
18351 b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	599 }
20458 16f9c0a46cbb dsync: When full resync is wanted in a stateful sync, output empty state. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20073 diff changeset	600 if (require_full_resync)
16f9c0a46cbb dsync: When full resync is wanted in a stateful sync, output empty state. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20073 diff changeset	601 brain->require_full_resync = TRUE;
18351 b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	602 brain->state = DSYNC_STATE_DONE;
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	603 return TRUE;
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	604 }
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	605
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	606 static bool dsync_brain_run_real(struct dsync_brain brain, bool changed_r)
9639 ffda7bd92ebc dsync: Skip syncing mailboxes whose uidvalidity/uidnext/highest-modseq hasn't changed. Timo Sirainen <tss@iki.fi> parents: 9638 diff changeset	607 {
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	608 enum dsync_state orig_state = brain->state;
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	609 enum dsync_box_state orig_box_recv_state = brain->box_recv_state;
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	610 enum dsync_box_state orig_box_send_state = brain->box_send_state;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	611 bool changed = FALSE, ret = TRUE;
9639 ffda7bd92ebc dsync: Skip syncing mailboxes whose uidvalidity/uidnext/highest-modseq hasn't changed. Timo Sirainen <tss@iki.fi> parents: 9638 diff changeset	612
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	613 if (brain->failed)
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	614 return FALSE;
9686 4d5cc6ce68aa dsync: Lots of improvements and fixes. Appears to be somewhat working now. Timo Sirainen <tss@iki.fi> parents: 9644 diff changeset	615
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	616 switch (brain->state) {
15782 6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	617 case DSYNC_STATE_MASTER_RECV_HANDSHAKE:
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	618 changed = dsync_brain_master_recv_handshake(brain);
6f024e0289da dsync: -l parameter locking is now done on the server with "lower" hostname. Timo Sirainen <tss@iki.fi> parents: 15750 diff changeset	619 break;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	620 case DSYNC_STATE_SLAVE_RECV_HANDSHAKE:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	621 changed = dsync_brain_slave_recv_handshake(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	622 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	623 case DSYNC_STATE_MASTER_SEND_LAST_COMMON:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	624 dsync_brain_master_send_last_common(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	625 changed = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	626 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	627 case DSYNC_STATE_SLAVE_RECV_LAST_COMMON:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	628 changed = dsync_brain_slave_recv_last_common(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	629 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	630 case DSYNC_STATE_SEND_MAILBOX_TREE:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	631 dsync_brain_send_mailbox_tree(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	632 changed = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	633 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	634 case DSYNC_STATE_RECV_MAILBOX_TREE:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	635 changed = dsync_brain_recv_mailbox_tree(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	636 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	637 case DSYNC_STATE_SEND_MAILBOX_TREE_DELETES:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	638 dsync_brain_send_mailbox_tree_deletes(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	639 changed = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	640 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	641 case DSYNC_STATE_RECV_MAILBOX_TREE_DELETES:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	642 changed = dsync_brain_recv_mailbox_tree_deletes(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	643 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	644 case DSYNC_STATE_MASTER_SEND_MAILBOX:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	645 dsync_brain_master_send_mailbox(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	646 changed = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	647 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	648 case DSYNC_STATE_SLAVE_RECV_MAILBOX:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	649 changed = dsync_brain_slave_recv_mailbox(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	650 break;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	651 case DSYNC_STATE_SYNC_MAILS:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	652 changed = dsync_brain_sync_mails(brain);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	653 break;
18351 b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	654 case DSYNC_STATE_FINISH:
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	655 changed = dsync_brain_finish(brain);
b4dbe64c0032 dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. Timo Sirainen <tss@iki.fi> parents: 18287 diff changeset	656 break;
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	657 case DSYNC_STATE_DONE:
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	658 changed = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	659 ret = FALSE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	660 break;
9639 ffda7bd92ebc dsync: Skip syncing mailboxes whose uidvalidity/uidnext/highest-modseq hasn't changed. Timo Sirainen <tss@iki.fi> parents: 9638 diff changeset	661 }
16539 b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	662 if (brain->verbose_proctitle) {
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	663 if (orig_state != brain->state \|\|
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	664 orig_box_recv_state != brain->box_recv_state \|\|
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	665 orig_box_send_state != brain->box_send_state \|\|
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	666 ++brain->proctitle_update_counter % 100 == 0)
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	667 process_title_set(dsync_brain_get_proctitle(brain));
b5876fa03b0e dsync: If verbose_proctitle=yes, show the current state in it. Timo Sirainen <tss@iki.fi> parents: 16538 diff changeset	668 }
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	669 *changed_r = changed;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	670 return brain->failed ? FALSE : ret;
10371 b99a19d5a93c dsync: Sync mailbox renames. Timo Sirainen <tss@iki.fi> parents: 10368 diff changeset	671 }
b99a19d5a93c dsync: Sync mailbox renames. Timo Sirainen <tss@iki.fi> parents: 10368 diff changeset	672
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	673 bool dsync_brain_run(struct dsync_brain brain, bool changed_r)
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	674 {
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	675 bool ret;
10373 036db604f86c dsync: Fixed problems with syncing mailbox names that are invalid on other side. Timo Sirainen <tss@iki.fi> parents: 10371 diff changeset	676
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	677 *changed_r = FALSE;
9559 34eef8a2716b dsync: Initial implementation. Timo Sirainen <tss@iki.fi> parents: diff changeset	678
15037 920756cd29b8 dsync: Renamed "slave" to "ibc" (= inter-brain communicator) Timo Sirainen <tss@iki.fi> parents: 14939 diff changeset	679 if (dsync_ibc_has_failed(brain->ibc)) {
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	680 brain->failed = TRUE;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	681 return FALSE;
12028 6ef21adeb61d dsync: If worker fails, abort earlier. Timo Sirainen <tss@iki.fi> parents: 11971 diff changeset	682 }
6ef21adeb61d dsync: If worker fails, abort earlier. Timo Sirainen <tss@iki.fi> parents: 11971 diff changeset	683
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	684 T_BEGIN {
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	685 ret = dsync_brain_run_real(brain, changed_r);
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	686 } T_END;
b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	687 return ret;
10368 9f2e8d230bd5 dsync: If some uid/modseq changes couldn't be done, exit with different value. Timo Sirainen <tss@iki.fi> parents: 10356 diff changeset	688 }
12124 eb5adb193064 dsync: Don't crash/hang if syncing fails before ioloop is started. Timo Sirainen <tss@iki.fi> parents: 12028 diff changeset	689
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	690 static void dsync_brain_mailbox_states_dump(struct dsync_brain *brain)
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	691 {
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	692 struct hash_iterate_context *iter;
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	693 struct dsync_mailbox_state *state;
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	694 uint8_t *guid;
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	695
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	696 iter = hash_table_iterate_init(brain->mailbox_states);
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	697 while (hash_table_iterate(iter, brain->mailbox_states, &guid, &state)) {
17282 9669c9a8984f dsync: Include messages_count in the mailbox states. Timo Sirainen <tss@iki.fi> parents: 17279 diff changeset	698 i_debug("brain %c: Mailbox %s state: uidvalidity=%u uid=%u modseq=%llu pvt_modseq=%llu messages=%u changes_during_sync=%d",
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	699 brain->master_brain ? 'M' : 'S',
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	700 guid_128_to_string(guid),
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	701 state->last_uidvalidity,
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	702 state->last_common_uid,
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	703 (unsigned long long)state->last_common_modseq,
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	704 (unsigned long long)state->last_common_pvt_modseq,
17282 9669c9a8984f dsync: Include messages_count in the mailbox states. Timo Sirainen <tss@iki.fi> parents: 17279 diff changeset	705 state->last_messages_count,
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	706 state->changes_during_sync);
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	707 }
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	708 hash_table_iterate_deinit(&iter);
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	709 }
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	710
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	711 void dsync_brain_get_state(struct dsync_brain brain, string_t output)
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	712 {
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	713 struct hash_iterate_context *iter;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	714 struct dsync_mailbox_node *node;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	715 const struct dsync_mailbox_state *new_state;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	716 struct dsync_mailbox_state *state;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	717 const uint8_t *guid_p;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	718 uint8_t *guid;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	719
20073 1b4e84d0a728 dsync: If full resync is requested, return empty state string. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19646 diff changeset	720 if (brain->require_full_resync)
1b4e84d0a728 dsync: If full resync is requested, return empty state string. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19646 diff changeset	721 return;
1b4e84d0a728 dsync: If full resync is requested, return empty state string. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19646 diff changeset	722
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	723 /* update mailbox states */
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	724 array_foreach(&brain->remote_mailbox_states, new_state) {
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	725 guid_p = new_state->mailbox_guid;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	726 state = hash_table_lookup(brain->mailbox_states, guid_p);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	727 if (state != NULL)
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	728 state = new_state;
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	729 else
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	730 dsync_mailbox_state_add(brain, new_state);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	731 }
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	732
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	733 /* remove nonexistent mailboxes */
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	734 iter = hash_table_iterate_init(brain->mailbox_states);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	735 while (hash_table_iterate(iter, brain->mailbox_states, &guid, &state)) {
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	736 node = dsync_mailbox_tree_lookup_guid(brain->local_mailbox_tree,
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	737 guid);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	738 if (node == NULL \|\|
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	739 node->existence != DSYNC_MAILBOX_NODE_EXISTS) {
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	740 if (brain->debug) {
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	741 i_debug("brain %c: Removed state for deleted mailbox %s",
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	742 brain->master_brain ? 'M' : 'S',
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	743 guid_128_to_string(guid));
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	744 }
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	745 hash_table_remove(brain->mailbox_states, guid);
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	746 }
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	747 }
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	748 hash_table_iterate_deinit(&iter);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	749
17271 bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	750 if (brain->debug) {
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	751 i_debug("brain %c: Exported mailbox states:",
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	752 brain->master_brain ? 'M' : 'S');
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	753 dsync_brain_mailbox_states_dump(brain);
bb5deb884303 dsync: Added more debug output Timo Sirainen <tss@iki.fi> parents: 17221 diff changeset	754 }
15232 6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	755 dsync_mailbox_states_export(brain->mailbox_states, output);
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	756 }
6c850258002f doveadm sync/backup: Added -s <state> parameter to do a fast stateful sync. Timo Sirainen <tss@iki.fi> parents: 15231 diff changeset	757
16253 1a58d4bcb49c dsync: -U parameter never updated replicator's full_sync state. Timo Sirainen <tss@iki.fi> parents: 16251 diff changeset	758 enum dsync_brain_sync_type dsync_brain_get_sync_type(struct dsync_brain *brain)
1a58d4bcb49c dsync: -U parameter never updated replicator's full_sync state. Timo Sirainen <tss@iki.fi> parents: 16251 diff changeset	759 {
1a58d4bcb49c dsync: -U parameter never updated replicator's full_sync state. Timo Sirainen <tss@iki.fi> parents: 16251 diff changeset	760 return brain->sync_type;
1a58d4bcb49c dsync: -U parameter never updated replicator's full_sync state. Timo Sirainen <tss@iki.fi> parents: 16251 diff changeset	761 }
1a58d4bcb49c dsync: -U parameter never updated replicator's full_sync state. Timo Sirainen <tss@iki.fi> parents: 16251 diff changeset	762
12124 eb5adb193064 dsync: Don't crash/hang if syncing fails before ioloop is started. Timo Sirainen <tss@iki.fi> parents: 12028 diff changeset	763 bool dsync_brain_has_failed(struct dsync_brain *brain)
eb5adb193064 dsync: Don't crash/hang if syncing fails before ioloop is started. Timo Sirainen <tss@iki.fi> parents: 12028 diff changeset	764 {
14584 b2076acc3715 Initial version of dsync rewrite. Timo Sirainen <tss@iki.fi> parents: 14253 diff changeset	765 return brain->failed;
12124 eb5adb193064 dsync: Don't crash/hang if syncing fails before ioloop is started. Timo Sirainen <tss@iki.fi> parents: 12028 diff changeset	766 }
16454 5593d6129712 dsync: If unexpected changes happened during sync, log a warning and exit with code 2. Timo Sirainen <tss@iki.fi> parents: 16398 diff changeset	767
21256 4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	768 const char dsync_brain_get_unexpected_changes_reason(struct dsync_brain brain,
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	769 bool *remote_only_r)
16454 5593d6129712 dsync: If unexpected changes happened during sync, log a warning and exit with code 2. Timo Sirainen <tss@iki.fi> parents: 16398 diff changeset	770 {
21256 4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	771 if (brain->changes_during_sync == NULL &&
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	772 brain->changes_during_remote_sync) {
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	773 *remote_only_r = TRUE;
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	774 return "Remote notified that changes happened during sync";
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	775 }
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	776 *remote_only_r = FALSE;
16454 5593d6129712 dsync: If unexpected changes happened during sync, log a warning and exit with code 2. Timo Sirainen <tss@iki.fi> parents: 16398 diff changeset	777 return brain->changes_during_sync;
5593d6129712 dsync: If unexpected changes happened during sync, log a warning and exit with code 2. Timo Sirainen <tss@iki.fi> parents: 16398 diff changeset	778 }
16540 84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	779
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	780 bool dsync_brain_want_namespace(struct dsync_brain *brain,
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	781 struct mail_namespace *ns)
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	782 {
17170 08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	783 struct mail_namespace const nsp;
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	784
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	785 if (array_is_created(&brain->sync_namespaces)) {
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	786 array_foreach(&brain->sync_namespaces, nsp) {
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	787 if (ns == *nsp)
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	788 return TRUE;
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	789 }
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	790 return FALSE;
08f1c7af0ac0 dsync: Support multiple -n parameters. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	791 }
16540 84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	792 if (ns->alias_for != NULL) {
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	793 /* always skip aliases */
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	794 return FALSE;
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	795 }
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	796 if (brain->sync_visible_namespaces) {
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	797 if ((ns->flags & NAMESPACE_FLAG_HIDDEN) == 0)
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	798 return TRUE;
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	799 if ((ns->flags & (NAMESPACE_FLAG_LIST_PREFIX \|
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	800 NAMESPACE_FLAG_LIST_CHILDREN)) != 0)
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	801 return TRUE;
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	802 return FALSE;
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	803 } else {
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	804 return strcmp(ns->unexpanded_set->location,
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	805 SETTING_STRVAR_UNEXPANDED) == 0;
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	806 }
84014270ee37 dsync: Don't try to find mailboxes from unwanted namespaces. Timo Sirainen <tss@iki.fi> parents: 16539 diff changeset	807 }
21256 4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	808
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	809 void dsync_brain_set_changes_during_sync(struct dsync_brain *brain,
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	810 const char *reason)
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	811 {
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	812 if (brain->debug) {
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	813 i_debug("brain %c: Change during sync: %s",
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	814 brain->master_brain ? 'M' : 'S', reason);
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	815 }
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	816 if (brain->changes_during_sync == NULL)
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	817 brain->changes_during_sync = p_strdup(brain->pool, reason);
4ac3461df334 dsync: When logging "Mailbox changed caused a desync", log also the reason. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20831 diff changeset	818 }

Mercurial > dovecot > core-2.2

annotate src/doveadm/dsync/dsync-brain.c @ 21256:4ac3461df334