dovecot/original-hg/dovecot-1.2: src/lib-index/mail-transaction-log.c annotate

annotate src/lib-index/mail-transaction-log.c @ 5236:6761b2a46ec3 HEAD

Fixed last change: Don't crash now when rotating non-locked log..

author	Timo Sirainen <tss@iki.fi>
date	Wed, 07 Mar 2007 02:47:54 +0200
parents	f107ddd56550
children	c1ef15c1a1fa

rev	line source
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1 /* Copyright (C) 2003-2004 Timo Sirainen */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
1976 4e0269adb24a Don't rotate transaction log too often Timo Sirainen <tss@iki.fi> parents: 1974 diff changeset	4 #include "ioloop.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "buffer.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	6 #include "file-dotlock.h"
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4023 diff changeset	7 #include "nfs-workarounds.h"
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	8 #include "close-keep-errno.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	9 #include "read-full.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	10 #include "write-full.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	11 #include "mmap-util.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	12 #include "mail-index-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	13 #include "mail-index-view-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	14 #include "mail-transaction-log-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	15 #include "mail-transaction-util.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	16 #include "mail-index-transaction-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	17
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	18 #include <stddef.h>
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	19 #include <stdio.h>
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	20 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	21
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	22 #define LOG_PREFETCH 1024
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	23
1920 d11efbf1d341 Added fcntl_lock_disable setting to allow indexes to work with NFS. Some Timo Sirainen <tss@iki.fi> parents: 1919 diff changeset	24 /* this lock should never exist for a long time.. */
3350 2af6d96e4134 Updated timeouts Timo Sirainen <tss@iki.fi> parents: 3342 diff changeset	25 #define LOG_DOTLOCK_TIMEOUT 60
3981 0d64f8888dcd Removed immediate_stale_timeout and changed the stale_timeout behavior to Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	26 #define LOG_DOTLOCK_STALE_TIMEOUT 60
1920 d11efbf1d341 Added fcntl_lock_disable setting to allow indexes to work with NFS. Some Timo Sirainen <tss@iki.fi> parents: 1919 diff changeset	27
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	28 #define MAIL_TRANSACTION_LOG_SUFFIX ".log"
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	29 #define LOG_NEW_DOTLOCK_SUFFIX ".newlock"
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	30
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	31 static struct mail_transaction_log_file *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	32 mail_transaction_log_file_open_or_create(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	33 const char *path);
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	34 static int
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	35 mail_transaction_log_file_read(struct mail_transaction_log_file *file,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	36 uoff_t offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	37
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	38 void
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	39 mail_transaction_log_file_set_corrupted(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	40 const char *fmt, ...)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	41 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	42 va_list va;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	43
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	44 file->hdr.indexid = 0;
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	45 if (!MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	46 if (pwrite_full(file->fd, &file->hdr.indexid,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	47 sizeof(file->hdr.indexid), 0) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	48 mail_index_file_set_syscall_error(file->log->index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	49 file->filepath, "pwrite()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	50 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	51 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	52
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	53 va_start(va, fmt);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	54 t_push();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	55 mail_index_set_error(file->log->index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	56 "Corrupted transaction log file %s: %s",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	57 file->filepath, t_strdup_vprintf(fmt, va));
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 t_pop();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	59 va_end(va);
2862 6a8c73bfbcad fsck index if transaction log is found to be corrupted. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	60
5213 2969edccb9b0 Don't try to fsck index file in error conditions if it's not yet opened. Timo Sirainen <tss@iki.fi> parents: 5089 diff changeset	61 if (file->log->index->log != NULL && file->log->index->map != NULL) {
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	62 /* this may have happened because of broken index.
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	63 make sure it's ok. */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	64 (void)mail_index_fsck(file->log->index);
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	65 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	66 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	67
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	68 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	69 mail_transaction_log_file_dotlock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	70 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	71 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	72
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	73 if (file->log->dotlock_count > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	74 ret = 1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	75 else {
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	76 ret = file_dotlock_create(&file->log->dotlock_settings,
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	77 file->filepath, 0,
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	78 &file->log->dotlock);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	79 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	80 if (ret > 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	81 file->log->dotlock_count++;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	82 file->locked = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	83 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	84 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	85 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	86 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	87 file->filepath,
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	88 "file_dotlock_create()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	89 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	90 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	91
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	92 mail_index_set_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	93 "Timeout while waiting for release of "
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	94 "dotlock for transaction log file %s",
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	95 file->filepath);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	96 file->log->index->index_lock_timeout = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	97 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	98 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	99
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	100 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	101 mail_transaction_log_file_undotlock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	102 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	103 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	104
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	105 if (--file->log->dotlock_count > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	106 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	107
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	108 ret = file_dotlock_delete(&file->log->dotlock);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	109 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	110 mail_index_file_set_syscall_error(file->log->index,
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	111 file->filepath, "file_dotlock_delete()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	112 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	113 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	114
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	115 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	116 mail_index_set_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	117 "Dotlock was lost for transaction log file %s",
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	118 file->filepath);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	119 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	120 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	121 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	122 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	123
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	124 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	125 mail_transaction_log_file_lock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	126 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	127 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	128
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	129 if (file->locked)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	130 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	131
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	132 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	133 file->locked = TRUE;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	134 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	135 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	136
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4828 diff changeset	137 if (file->log->index->lock_method == FILE_LOCK_METHOD_DOTLOCK)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	138 return mail_transaction_log_file_dotlock(file);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	139
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4828 diff changeset	140 i_assert(file->file_lock == NULL);
2941 89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	141 ret = mail_index_lock_fd(file->log->index, file->filepath, file->fd,
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4828 diff changeset	142 F_WRLCK, MAIL_INDEX_LOCK_SECS,
f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4828 diff changeset	143 &file->file_lock);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	144 if (ret > 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	145 file->locked = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	146 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	147 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	148 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	149 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	150 file->filepath,
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	151 "mail_index_wait_lock_fd()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	152 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	153 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	154
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	155 mail_index_set_error(file->log->index,
3552 6a1ea27a4256 More error message fixing Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	156 "Timeout while waiting for lock for transaction log file %s",
6a1ea27a4256 More error message fixing Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	157 file->filepath);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	158 file->log->index->index_lock_timeout = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	159 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	160 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	161
3146 9a8bf1c86699 Moved code into mail-transaction-log-append.c Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	162 void mail_transaction_log_file_unlock(struct mail_transaction_log_file *file)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	163 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	164 if (!file->locked)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	165 return;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	166
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	167 file->locked = FALSE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	168
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	169 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	170 return;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	171
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4828 diff changeset	172 if (file->log->index->lock_method == FILE_LOCK_METHOD_DOTLOCK) {
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	173 mail_transaction_log_file_undotlock(file);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	174 return;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	175 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	176
4876 f1d77064884c Lock handling changes. Everything goes through file-lock API now and there's Timo Sirainen <tss@iki.fi> parents: 4828 diff changeset	177 file_unlock(&file->file_lock);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	178 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	179
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	180 #define INDEX_HAS_MISSING_LOGS(index, file) \
1952 05f76fe32f0d Don't crash in assert() with corrupted index file. Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	181 !(((file)->hdr.file_seq == (index)->hdr->log_file_seq && \
3170 42e1e9dbf19e hdr_size fix Timo Sirainen <tss@iki.fi> parents: 3169 diff changeset	182 (index)->hdr->log_file_int_offset >= (file)->hdr.hdr_size) \|\| \
1952 05f76fe32f0d Don't crash in assert() with corrupted index file. Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	183 ((file)->hdr.prev_file_seq == (index)->hdr->log_file_seq && \
2819 bea9c88c8763 Use separate sync offsets for internal/external transactions. Pending external Timo Sirainen <tss@iki.fi> parents: 2815 diff changeset	184 (file)->hdr.prev_file_offset == (index)->hdr->log_file_int_offset))
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	185
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	186 static int mail_transaction_log_check_file_seq(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	187 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	188 struct mail_index *index = log->index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	189 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	190 unsigned int lock_id;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	191 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	192
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	193 if (mail_transaction_log_lock_head(log) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	194 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	195
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	196 file = log->head;
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	197 file->refcount++;
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	198
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	199 ret = mail_index_lock_shared(index, TRUE, &lock_id);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	200 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	201 ret = mail_index_map(index, FALSE);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	202 if (ret <= 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	203 ret = -1;
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	204 else if (INDEX_HAS_MISSING_LOGS(index, file)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	205 /* broken - fix it by creating a new log file */
2815 ed85b5a62e06 fix Timo Sirainen <tss@iki.fi> parents: 2762 diff changeset	206 ret = mail_transaction_log_rotate(log, FALSE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	207 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	208 }
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	209
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	210 if (--file->refcount == 0)
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	211 mail_transaction_logs_clean(log);
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	212 else
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	213 mail_transaction_log_file_unlock(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	214 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	215 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	216
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	217 struct mail_transaction_log *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	218 mail_transaction_log_open_or_create(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	219 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	220 struct mail_transaction_log *log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	221 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	222
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	223 log = i_new(struct mail_transaction_log, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	224 log->index = index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	225
4968 1baf9dd3fc40 Added dotlock_use_excl setting. Timo Sirainen <tss@iki.fi> parents: 4876 diff changeset	226 log->dotlock_settings.use_excl_lock = index->use_excl_dotlocks;
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	227 log->dotlock_settings.timeout = LOG_DOTLOCK_TIMEOUT;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	228 log->dotlock_settings.stale_timeout = LOG_DOTLOCK_STALE_TIMEOUT;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	229
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	230 log->new_dotlock_settings = log->dotlock_settings;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	231 log->new_dotlock_settings.lock_suffix = LOG_NEW_DOTLOCK_SUFFIX;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	232
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	233 path = t_strconcat(log->index->filepath,
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	234 MAIL_TRANSACTION_LOG_SUFFIX, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	235 log->head = mail_transaction_log_file_open_or_create(log, path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	236 if (log->head == NULL) {
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	237 /* fallback to in-memory indexes */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	238 if (mail_index_move_to_memory(index) < 0) {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	239 mail_transaction_log_close(&log);
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	240 return NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	241 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	242 log->head = mail_transaction_log_file_open_or_create(log, path);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	243 i_assert(log->head != NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	244 }
4828 7cc1b8fec8ab Reference counting fixes. Fixes an assert crash when closing the index. Timo Sirainen <tss@iki.fi> parents: 4785 diff changeset	245 log->head->refcount++;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	246
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	247 if (index->fd != -1 &&
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	248 INDEX_HAS_MISSING_LOGS(index, log->head)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	249 /* head log file isn't same as head index file -
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	250 shouldn't happen except in race conditions.
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	251 lock them and check again */
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	252 if (mail_transaction_log_check_file_seq(log) < 0) {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	253 mail_transaction_log_close(&log);
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	254 return NULL;
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	255 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	256 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	257 return log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	258 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	259
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	260 void mail_transaction_log_close(struct mail_transaction_log **_log)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	261 {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	262 struct mail_transaction_log log = _log;
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	263
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1941 diff changeset	264 mail_transaction_log_views_close(log);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	265
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	266 if (log->head != NULL)
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	267 log->head->refcount--;
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	268 mail_transaction_logs_clean(log);
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	269 i_assert(log->files == NULL);
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	270
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	271 *_log = NULL;
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1941 diff changeset	272 log->index->log = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	273 i_free(log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	274 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	275
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	276 static void
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	277 mail_transaction_log_file_free(struct mail_transaction_log_file *file)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	278 {
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	279 struct mail_transaction_log_file **p;
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	280 int old_errno = errno;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	281
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	282 mail_transaction_log_file_unlock(file);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	283
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	284 for (p = &file->log->files; p != NULL; p = &(p)->next) {
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	285 if (*p == file) {
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	286 *p = file->next;
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	287 break;
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	288 }
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	289 }
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	290
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	291 if (file == file->log->head)
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	292 file->log->head = NULL;
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	293
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	294 if (file->buffer != NULL)
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	295 buffer_free(file->buffer);
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	296
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	297 if (file->mmap_base != NULL) {
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	298 if (munmap(file->mmap_base, file->mmap_size) < 0) {
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	299 mail_index_file_set_syscall_error(file->log->index,
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	300 file->filepath,
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	301 "munmap()");
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	302 }
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	303 }
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	304
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	305 if (file->fd != -1) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	306 if (close(file->fd) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	307 mail_index_file_set_syscall_error(file->log->index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	308 file->filepath,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	309 "close()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	310 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	311 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	312
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	313 i_free(file->filepath);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	314 i_free(file);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	315
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	316 errno = old_errno;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	317 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	318
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	319 int mail_transaction_log_move_to_memory(struct mail_transaction_log *log)
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	320 {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	321 struct mail_transaction_log_file *file = log->head;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	322
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	323 if (file == NULL \|\| MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	324 return 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	325
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	326 /* read the whole file to memory. we might currently be appending
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	327 data into it, so we want to read it up to end of file */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	328 file->buffer_offset = 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	329
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	330 if (file->buffer != NULL) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	331 buffer_free(file->buffer);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	332 file->buffer = NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	333 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	334
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	335 if (file->mmap_base != NULL) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	336 if (munmap(file->mmap_base, file->mmap_size) < 0) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	337 mail_index_file_set_syscall_error(file->log->index,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	338 file->filepath,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	339 "munmap()");
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	340 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	341 file->mmap_base = NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	342 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	343
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	344 if (mail_transaction_log_file_read(file, 0) <= 0)
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	345 return -1;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	346
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	347 /* after we've read the file into memory, make it into in-memory
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	348 log file */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	349 if (close(file->fd) < 0) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	350 mail_index_file_set_syscall_error(file->log->index,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	351 file->filepath, "close()");
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	352 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	353 file->fd = -1;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	354 return 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	355 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	356
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	357 static int
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	358 mail_transaction_log_file_read_hdr(struct mail_transaction_log_file *file,
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	359 int head, bool ignore_estale)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	360 {
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	361 struct mail_transaction_log_file *f;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	362 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	363
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	364 i_assert(!MAIL_INDEX_IS_IN_MEMORY(file->log->index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	365
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	366 ret = pread_full(file->fd, &file->hdr, sizeof(file->hdr), 0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	367 if (ret < 0) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	368 if (errno != ESTALE \|\| !ignore_estale) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	369 mail_index_file_set_syscall_error(file->log->index,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	370 file->filepath,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	371 "pread_full()");
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	372 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	373 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	374 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	375 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	376 mail_transaction_log_file_set_corrupted(file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	377 "unexpected end of file while reading header");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	378 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	379 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	380
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	381 if (file->hdr.major_version != MAIL_TRANSACTION_LOG_MAJOR_VERSION) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	382 /* incompatible version - fix silently */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	383 return 0;
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	384 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	385 if (file->hdr.hdr_size < MAIL_TRANSACTION_LOG_HEADER_MIN_SIZE) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	386 mail_transaction_log_file_set_corrupted(file,
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	387 "Header size too small");
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	388 return 0;
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	389 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	390 if (file->hdr.hdr_size < sizeof(file->hdr)) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	391 /* @UNSAFE: smaller than we expected - zero out the fields we
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	392 shouldn't have filled */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	393 memset(PTR_OFFSET(&file->hdr, file->hdr.hdr_size), 0,
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	394 sizeof(file->hdr) - file->hdr.hdr_size);
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	395 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	396
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	397 if (file->hdr.indexid == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	398 /* corrupted */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	399 mail_index_set_error(file->log->index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	400 "Transaction log file %s: marked corrupted",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	401 file->filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	402 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	403 }
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	404 if (file->hdr.indexid != file->log->index->indexid) {
3556 6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	405 if (file->log->index->fd != -1) {
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	406 /* index file was probably just rebuilt and we don't
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	407 know about it yet */
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	408 mail_transaction_log_file_set_corrupted(file,
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	409 "invalid indexid (%u != %u)",
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	410 file->hdr.indexid, file->log->index->indexid);
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	411 return 0;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	412 }
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	413
3556 6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	414 /* creating index file. since transaction log is created
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	415 first, use the indexid in it to create the main index
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	416 to avoid races. */
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	417 file->log->index->indexid = file->hdr.indexid;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	418 }
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	419
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	420 /* make sure we already don't have a file with the same sequence
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	421 opened. it shouldn't happen unless the old log file was
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	422 corrupted.
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	423
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	424 If we're opening head log file, make sure the sequence is larger
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	425 than any existing one. */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	426 if (head) {
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	427 for (f = file->log->files; f != NULL; f = f->next) {
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	428 if (f->hdr.file_seq >= file->hdr.file_seq) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	429 mail_transaction_log_file_set_corrupted(file,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	430 "invalid new transaction log sequence "
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	431 "(%u >= %u)",
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	432 f->hdr.file_seq, file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	433 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	434 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	435 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	436 } else {
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	437 for (f = file->log->files; f != NULL; f = f->next) {
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	438 if (f->hdr.file_seq == file->hdr.file_seq) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	439 mail_transaction_log_file_set_corrupted(file,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	440 "old transaction log already opened "
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	441 "(%u == %u)",
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	442 f->hdr.file_seq, file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	443 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	444 }
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	445 }
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	446 }
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	447
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	448 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	449 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	450
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	451 static int
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	452 mail_transaction_log_init_hdr(struct mail_transaction_log *log,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	453 struct mail_transaction_log_header *hdr)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	454 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	455 struct mail_index *index = log->index;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	456 unsigned int lock_id;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	457
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	458 memset(hdr, 0, sizeof(*hdr));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	459 hdr->major_version = MAIL_TRANSACTION_LOG_MAJOR_VERSION;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	460 hdr->minor_version = MAIL_TRANSACTION_LOG_MINOR_VERSION;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	461 hdr->hdr_size = sizeof(struct mail_transaction_log_header);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	462 hdr->indexid = log->index->indexid;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	463 hdr->create_stamp = ioloop_time;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	464
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	465 if (index->fd != -1) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	466 /* not creating index - make sure we have latest header */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	467 if (mail_index_lock_shared(index, TRUE, &lock_id) < 0)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	468 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	469 if (mail_index_map(index, FALSE) <= 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	470 mail_index_unlock(index, lock_id);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	471 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	472 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	473 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	474 hdr->prev_file_seq = index->hdr->log_file_seq;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	475 hdr->prev_file_offset = index->hdr->log_file_int_offset;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	476 hdr->file_seq = index->hdr->log_file_seq+1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	477
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	478 if (index->fd != -1)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	479 mail_index_unlock(index, lock_id);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	480
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	481 if (log->head != NULL && hdr->file_seq <= log->head->hdr.file_seq) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	482 /* make sure the sequence grows */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	483 hdr->file_seq = log->head->hdr.file_seq+1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	484 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	485 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	486 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	487
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	488 static int
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	489 mail_transaction_log_file_create2(struct mail_transaction_log_file *file,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	490 bool lock, int new_fd,
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	491 struct dotlock **dotlock,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	492 dev_t dev, ino_t ino, uoff_t file_size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	493 {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	494 struct mail_index *index = file->log->index;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	495 struct mail_transaction_log_header hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	496 struct stat st;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	497 const char *path2;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	498 int old_fd, ret;
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	499 bool rename_existing;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	500
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	501 i_assert(!lock \|\| file->log->head->locked);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	502
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	503 /* log creation is locked now - see if someone already created it */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	504 if (lock) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	505 /* don't even bother checking the existing file, but rename it
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	506 if it exists */
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	507 rename_existing = TRUE;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	508 } else if ((old_fd = nfs_safe_open(file->filepath, O_RDWR)) != -1) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	509 if ((ret = fstat(old_fd, &st)) < 0) {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	510 mail_index_file_set_syscall_error(index, file->filepath,
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	511 "fstat()");
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	512 } else if (st.st_ino == ino && CMP_DEV_T(st.st_dev, dev) &&
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	513 (uoff_t)st.st_size == file_size) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	514 /* same file, still broken */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	515 } else {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	516 /* file changed, use the new file */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	517 (void)file_dotlock_delete(dotlock);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	518 file->fd = old_fd;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	519 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	520 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	521
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	522 (void)close(old_fd);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	523 old_fd = -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	524
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	525 if (ret < 0) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	526 /* fstat() failure, return after closing fd.. */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	527 return -1;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	528 }
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	529 rename_existing = TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	530 } else if (errno != ENOENT) {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	531 mail_index_file_set_syscall_error(index, file->filepath,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	532 "open()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	533 return -1;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	534 } else {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	535 rename_existing = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	536 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	537
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	538 if (mail_transaction_log_init_hdr(file->log, &hdr) < 0)
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	539 return -1;
1936 bb1bde2680b0 assert Timo Sirainen <tss@iki.fi> parents: 1935 diff changeset	540
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	541 if (write_full(new_fd, &hdr, sizeof(hdr)) < 0) {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	542 mail_index_file_set_syscall_error(index, file->filepath,
1974 d85f71ffeb8f file_dotlock_open/delete/replace now has lock_suffix parameter. NULL Timo Sirainen <tss@iki.fi> parents: 1967 diff changeset	543 "write_full()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	544 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	545 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	546
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	547 if (lock) {
5015 aa4ff7a9c3f7 And now really fix the log rotation race. Timo Sirainen <tss@iki.fi> parents: 5013 diff changeset	548 file->fd = new_fd;
aa4ff7a9c3f7 And now really fix the log rotation race. Timo Sirainen <tss@iki.fi> parents: 5013 diff changeset	549 ret = mail_transaction_log_file_lock(file);
aa4ff7a9c3f7 And now really fix the log rotation race. Timo Sirainen <tss@iki.fi> parents: 5013 diff changeset	550 file->fd = -1;
aa4ff7a9c3f7 And now really fix the log rotation race. Timo Sirainen <tss@iki.fi> parents: 5013 diff changeset	551 if (ret < 0)
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	552 return -1;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	553 }
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	554
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	555 /* keep two log files */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	556 if (rename_existing) {
5021 623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	557 /* rename() would be nice and easy way to do this, except then
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	558 there's a race condition between the rename and
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	559 file_dotlock_replace(). during that time the log file
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	560 doesn't exist, which could cause problems. */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	561 path2 = t_strconcat(file->filepath, ".2", NULL);
5021 623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	562 if (unlink(path2) < 0 && errno != ENOENT) {
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	563 mail_index_set_error(index, "unlink(%s) failed: %m",
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	564 path2);
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	565 /* try to link() anyway */
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	566 }
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	567 if (link(file->filepath, path2) < 0 &&
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	568 errno != ENOENT && errno != EEXIST) {
623ed4a5339b Don't rename() .log to .log.2, because it temporarily loses the log file. Timo Sirainen <tss@iki.fi> parents: 5017 diff changeset	569 mail_index_set_error(index, "link(%s, %s) failed: %m",
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	570 file->filepath, path2);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	571 /* ignore the error. we don't care that much about the
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	572 second log file and we're going to overwrite this
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	573 first one. */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	574 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	575 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	576
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	577 if (file_dotlock_replace(dotlock,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	578 DOTLOCK_REPLACE_FLAG_DONT_CLOSE_FD) <= 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	579 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	580
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	581 /* success */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	582 file->fd = new_fd;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	583 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	584 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	585
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	586 static int
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	587 mail_transaction_log_file_create(struct mail_transaction_log_file *file,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	588 bool lock, dev_t dev, ino_t ino,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	589 uoff_t file_size)
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	590 {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	591 struct mail_index *index = file->log->index;
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	592 struct dotlock *dotlock;
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	593 struct stat st;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	594 mode_t old_mask;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	595 int fd;
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	596
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	597 i_assert(!MAIL_INDEX_IS_IN_MEMORY(index));
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	598
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	599 if (stat(index->dir, &st) < 0) {
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	600 if (ENOTFOUND(errno)) {
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	601 /* the whole index directory was deleted, which means
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	602 the mailbox was deleted by another process.
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	603 fail silently. */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	604 mail_index_mark_corrupted(index);
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	605 return -1;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	606 }
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	607 mail_index_file_set_syscall_error(index, index->dir, "stat()");
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	608 return -1;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	609 }
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	610
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	611 /* With dotlocking we might already have path.lock created, so this
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	612 filename has to be different. */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	613 old_mask = umask(index->mode ^ 0666);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	614 fd = file_dotlock_open(&file->log->new_dotlock_settings,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	615 file->filepath, 0, &dotlock);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	616 umask(old_mask);
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	617
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	618 if (fd == -1) {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	619 mail_index_file_set_syscall_error(index, file->filepath,
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	620 "file_dotlock_open()");
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	621 return -1;
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	622 }
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	623
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	624 if (index->gid != (gid_t)-1 &&
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	625 fchown(fd, (uid_t)-1, index->gid) < 0) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	626 mail_index_file_set_syscall_error(index, file->filepath,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	627 "fchown()");
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	628 (void)file_dotlock_delete(&dotlock);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	629 return -1;
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	630 }
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	631
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	632 /* either fd gets used or the dotlock gets deleted and returned fd
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	633 is for the existing file */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	634 if (mail_transaction_log_file_create2(file, lock, fd, &dotlock,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	635 dev, ino, file_size) < 0) {
3317 a19268b76093 Don't crash if dotlock replacing failed. Timo Sirainen <tss@iki.fi> parents: 3316 diff changeset	636 if (dotlock != NULL)
a19268b76093 Don't crash if dotlock replacing failed. Timo Sirainen <tss@iki.fi> parents: 3316 diff changeset	637 (void)file_dotlock_delete(&dotlock);
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	638 return -1;
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	639 }
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	640 return 0;
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	641 }
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	642
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	643 static void
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	644 mail_transaction_log_file_add_to_head(struct mail_transaction_log_file *file)
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	645 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	646 struct mail_transaction_log *log = file->log;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	647 struct mail_transaction_log_file **p;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	648
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	649 if (log->index->map != NULL &&
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	650 file->hdr.file_seq == log->index->map->hdr.log_file_seq &&
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	651 log->index->map->hdr.log_file_int_offset != 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	652 /* we can get a valid log offset from index file. initialize
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	653 sync_offset from it so we don't have to read the whole log
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	654 file from beginning. */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	655 file->sync_offset = log->index->map->hdr.log_file_int_offset;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	656 } else {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	657 file->sync_offset = file->hdr.hdr_size;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	658 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	659
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	660 /* append to end of list. */
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	661 for (p = &log->files; p != NULL; p = &(p)->next)
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	662 i_assert((*p)->hdr.file_seq < file->hdr.file_seq);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	663 *p = file;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	664 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	665
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	666 static void
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	667 mail_transaction_log_file_add_to_list(struct mail_transaction_log_file *file)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	668 {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	669 struct mail_transaction_log *log = file->log;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	670 struct mail_transaction_log_file **p;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	671
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	672 file->sync_offset = file->hdr.hdr_size;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	673
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	674 /* insert it to correct position */
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	675 for (p = &log->files; p != NULL; p = &(p)->next) {
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	676 i_assert((*p)->hdr.file_seq != file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	677 if ((*p)->hdr.file_seq > file->hdr.file_seq)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	678 break;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	679 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	680
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	681 file->next = *p;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	682 *p = file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	683 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	684
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	685 static struct mail_transaction_log_file *
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	686 mail_transaction_log_file_alloc(struct mail_transaction_log *log,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	687 const char *path)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	688 {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	689 struct mail_transaction_log_file *file;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	690
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	691 file = i_new(struct mail_transaction_log_file, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	692 file->log = log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	693 file->filepath = i_strdup(path);
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	694 file->fd = -1;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	695 return file;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	696 }
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	697
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	698 static int
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	699 mail_transaction_log_file_fd_open(struct mail_transaction_log_file *file,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	700 bool head, bool ignore_estale)
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	701 {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	702 struct stat st;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	703
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	704 i_assert(!MAIL_INDEX_IS_IN_MEMORY(file->log->index));
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	705
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	706 if (fstat(file->fd, &st) < 0) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	707 if (errno != ESTALE \|\| !ignore_estale) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	708 mail_index_file_set_syscall_error(file->log->index,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	709 file->filepath,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	710 "fstat()");
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	711 }
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	712 return -1;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	713 }
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	714
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	715 file->st_dev = st.st_dev;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	716 file->st_ino = st.st_ino;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	717 file->last_mtime = st.st_mtime;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	718 file->last_size = st.st_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	719
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	720 return mail_transaction_log_file_read_hdr(file, head, ignore_estale);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	721 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	722
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	723 static int
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	724 mail_transaction_log_file_fd_open_or_create(struct mail_transaction_log_file
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	725 *file, bool try_retry)
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	726 {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	727 int ret;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	728
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	729 ret = mail_transaction_log_file_fd_open(file, TRUE, !try_retry);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	730 if (ret == 0) {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	731 /* corrupted header, recreate the file */
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	732 if (mail_transaction_log_file_create(file, FALSE,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	733 file->st_dev,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	734 file->st_ino,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	735 file->last_size) < 0)
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	736 ret = -1;
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	737 else {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	738 ret = mail_transaction_log_file_fd_open(file, TRUE,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	739 FALSE);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	740 if (ret == 0) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	741 /* newly created transaction log corrupted */
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	742 return -1;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	743 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	744 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	745 }
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	746 if (ret < 0)
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	747 return errno == ENOENT && try_retry ? 0 : -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	748
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	749 mail_transaction_log_file_add_to_head(file);
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	750 return 1;
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	751 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	752
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	753 static struct mail_transaction_log_file *
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	754 mail_transaction_log_file_alloc_in_memory(struct mail_transaction_log *log)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	755 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	756 struct mail_transaction_log_file *file;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	757
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	758 file = i_new(struct mail_transaction_log_file, 1);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	759 file->log = log;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	760 file->filepath = i_strdup("(in-memory transaction log file)");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	761 file->fd = -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	762
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	763 if (mail_transaction_log_init_hdr(log, &file->hdr) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	764 i_free(file);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	765 return NULL;
2449 1543762353aa Optimization for setting initial sync_offset from index. Timo Sirainen <tss@iki.fi> parents: 2440 diff changeset	766 }
1543762353aa Optimization for setting initial sync_offset from index. Timo Sirainen <tss@iki.fi> parents: 2440 diff changeset	767
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	768 file->buffer = buffer_create_dynamic(default_pool, 4096);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	769 file->buffer_offset = sizeof(file->hdr);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	770
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	771 mail_transaction_log_file_add_to_head(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	772 return file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	773 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	774
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	775 static struct mail_transaction_log_file *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	776 mail_transaction_log_file_open_or_create(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	777 const char *path)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	778 {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	779 struct mail_transaction_log_file *file;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	780 unsigned int i;
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	781 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	782
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	783 if (MAIL_INDEX_IS_IN_MEMORY(log->index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	784 return mail_transaction_log_file_alloc_in_memory(log);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	785
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	786 file = mail_transaction_log_file_alloc(log, path);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	787
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	788 for (i = 0; ; i++) {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	789 file->fd = nfs_safe_open(path, O_RDWR);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	790 if (file->fd == -1) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	791 if (errno != ENOENT) {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	792 mail_index_file_set_syscall_error(log->index,
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	793 path,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	794 "open()");
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	795 break;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	796 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	797
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	798 /* doesn't exist, try creating it */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	799 if (mail_transaction_log_file_create(file, FALSE,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	800 0, 0, 0) < 0)
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	801 break;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	802 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	803
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	804 ret = mail_transaction_log_file_fd_open_or_create(file,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	805 i == MAIL_INDEX_ESTALE_RETRY_COUNT);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	806 if (ret > 0)
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	807 return file;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	808 if (ret < 0)
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	809 break;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	810
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	811 /* ESTALE - retry */
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	812 }
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	813
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	814 mail_transaction_log_file_free(file);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	815 return NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	816 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	817
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	818 static struct mail_transaction_log_file *
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	819 mail_transaction_log_file_open(struct mail_transaction_log *log,
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	820 const char *path)
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	821 {
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	822 struct mail_transaction_log_file *file;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	823 unsigned int i;
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	824 int ret;
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	825
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	826 file = mail_transaction_log_file_alloc(log, path);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	827 for (i = 0;; i++) {
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	828 file->fd = nfs_safe_open(path, O_RDWR);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	829 if (file->fd == -1) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	830 mail_index_file_set_syscall_error(log->index, path,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	831 "open()");
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	832 break;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	833 }
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	834
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	835 ret = mail_transaction_log_file_fd_open(file,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	836 TRUE, i < MAIL_INDEX_ESTALE_RETRY_COUNT);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	837 if (ret > 0) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	838 /* success */
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	839 mail_transaction_log_file_add_to_head(file);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	840 return file;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	841 }
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	842
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	843 if (ret == 0) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	844 /* corrupted */
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	845 break;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	846 }
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	847 if (errno != ESTALE \|\|
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	848 i == MAIL_INDEX_ESTALE_RETRY_COUNT) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	849 /* syscall error */
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	850 break;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	851 }
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	852
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	853 /* ESTALE - try again */
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	854 }
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	855
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	856 mail_transaction_log_file_free(file);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	857 return NULL;
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	858 }
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	859
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	860 void mail_transaction_logs_clean(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	861 {
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	862 struct mail_transaction_log_file file, next;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	863
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	864 for (file = log->files; file != NULL; file = next) {
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	865 next = file->next;
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	866
4828 7cc1b8fec8ab Reference counting fixes. Fixes an assert crash when closing the index. Timo Sirainen <tss@iki.fi> parents: 4785 diff changeset	867 i_assert(file->refcount >= 0);
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	868 if (file->refcount == 0)
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	869 mail_transaction_log_file_free(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	870 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	871 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	872
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	873 int mail_transaction_log_rotate(struct mail_transaction_log *log, bool lock)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	874 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	875 struct mail_transaction_log_file *file;
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	876 const char *path = log->head->filepath;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	877 struct stat st;
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	878 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	879
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	880 i_assert(log->head->locked);
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	881
5234 f107ddd56550 Don't crash when rotating locked in-memory log file. Timo Sirainen <tss@iki.fi> parents: 5213 diff changeset	882 if (MAIL_INDEX_IS_IN_MEMORY(log->index)) {
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	883 file = mail_transaction_log_file_alloc_in_memory(log);
5236 6761b2a46ec3 Fixed last change: Don't crash now when rotating non-locked log.. Timo Sirainen <tss@iki.fi> parents: 5234 diff changeset	884 if (lock)
6761b2a46ec3 Fixed last change: Don't crash now when rotating non-locked log.. Timo Sirainen <tss@iki.fi> parents: 5234 diff changeset	885 file->locked = TRUE;
5234 f107ddd56550 Don't crash when rotating locked in-memory log file. Timo Sirainen <tss@iki.fi> parents: 5213 diff changeset	886 } else {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	887 /* we're locked, we shouldn't need to worry about ESTALE
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	888 problems in here. */
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	889 if (fstat(log->head->fd, &st) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	890 mail_index_file_set_syscall_error(log->index, path,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	891 "fstat()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	892 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	893 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	894
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	895 file = mail_transaction_log_file_alloc(log, path);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	896 if (mail_transaction_log_file_create(file, lock, st.st_dev,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	897 st.st_ino,
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	898 st.st_size) < 0) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	899 mail_transaction_log_file_free(file);
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	900 return -1;
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	901 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	902
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	903 ret = mail_transaction_log_file_fd_open_or_create(file, FALSE);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	904 if (ret <= 0) {
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	905 i_assert(ret != 0);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	906 mail_transaction_log_file_free(file);
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	907 return -1;
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	908 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	909 }
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	910
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	911 i_assert(file->locked == lock);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	912
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	913 if (--log->head->refcount == 0)
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	914 mail_transaction_logs_clean(log);
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	915 else
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	916 mail_transaction_log_file_unlock(log->head);
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	917
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	918 i_assert(log->head != file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	919 log->head = file;
4828 7cc1b8fec8ab Reference counting fixes. Fixes an assert crash when closing the index. Timo Sirainen <tss@iki.fi> parents: 4785 diff changeset	920 log->head->refcount++;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	921 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	922 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	923
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	924 static int mail_transaction_log_refresh(struct mail_transaction_log *log,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	925 bool create_if_needed)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	926 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	927 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	928 struct stat st;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	929 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	930
5089 18559c7e3a15 Cleanups and minor fixes Timo Sirainen <tss@iki.fi> parents: 5021 diff changeset	931 i_assert(log->head != NULL);
18559c7e3a15 Cleanups and minor fixes Timo Sirainen <tss@iki.fi> parents: 5021 diff changeset	932
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	933 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(log->head))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	934 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	935
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	936 path = t_strconcat(log->index->filepath,
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	937 MAIL_TRANSACTION_LOG_SUFFIX, NULL);
4225 abdef9d86a94 Added nfs_safe_stat() to transparently work around ESTALE with stat(). Timo Sirainen <timo.sirainen@movial.fi> parents: 4072 diff changeset	938 if (nfs_safe_stat(path, &st) < 0) {
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	939 if (errno != ENOENT) {
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	940 mail_index_file_set_syscall_error(log->index, path,
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	941 "stat()");
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	942 return -1;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	943 }
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	944 /* log was deleted. just reopen/recreate it. */
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	945 } else {
5089 18559c7e3a15 Cleanups and minor fixes Timo Sirainen <tss@iki.fi> parents: 5021 diff changeset	946 if (log->head->st_ino == st.st_ino &&
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	947 CMP_DEV_T(log->head->st_dev, st.st_dev)) {
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	948 /* same file */
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	949 return 0;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	950 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	951 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	952
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	953 file = create_if_needed ?
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	954 mail_transaction_log_file_open_or_create(log, path) :
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	955 mail_transaction_log_file_open(log, path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	956 if (file == NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	957 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	958
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	959 i_assert(!file->locked);
2095 92d8b70829c8 fixes Timo Sirainen <tss@iki.fi> parents: 2073 diff changeset	960
5089 18559c7e3a15 Cleanups and minor fixes Timo Sirainen <tss@iki.fi> parents: 5021 diff changeset	961 if (--log->head->refcount == 0)
18559c7e3a15 Cleanups and minor fixes Timo Sirainen <tss@iki.fi> parents: 5021 diff changeset	962 mail_transaction_logs_clean(log);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	963
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	964 log->head = file;
4828 7cc1b8fec8ab Reference counting fixes. Fixes an assert crash when closing the index. Timo Sirainen <tss@iki.fi> parents: 4785 diff changeset	965 log->head->refcount++;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	966 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	967 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	968
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	969 int mail_transaction_log_file_find(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	970 uint32_t file_seq,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	971 struct mail_transaction_log_file **file_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	972 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	973 struct mail_transaction_log_file *file;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	974 struct stat st;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	975 const char *path;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	976 int ret, fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	977
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	978 if (file_seq > log->head->hdr.file_seq) {
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	979 /* don't try to recreate log file if it gets lost. we're
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	980 already in trouble and with mmap_disable the creation
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	981 could cause a recursive mail_index_map() call */
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	982 if (mail_transaction_log_refresh(log, FALSE) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	983 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	984 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	985
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	986 for (file = log->files; file != NULL; file = file->next) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	987 if (file->hdr.file_seq == file_seq) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	988 *file_r = file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	989 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	990 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	991 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	992
5017 987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	993 if (MAIL_INDEX_IS_IN_MEMORY(log->index))
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	994 return 0;
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	995
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	996 /* see if we have it in log.2 file */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	997 path = t_strconcat(log->index->filepath,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	998 MAIL_TRANSACTION_LOG_SUFFIX".2", NULL);
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4023 diff changeset	999 fd = nfs_safe_open(path, O_RDWR);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1000 if (fd == -1) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1001 if (errno == ENOENT)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1002 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1003
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1004 mail_index_file_set_syscall_error(log->index, path, "open()");
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1005 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1006 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1007
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1008 if (fstat(fd, &st) < 0) {
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1009 close_keep_errno(fd);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1010 if (errno == ESTALE) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1011 /* treat as "doesn't exist" */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1012 return 0;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1013 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1014 mail_index_file_set_syscall_error(log->index, path, "fstat()");
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1015 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1016 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1017
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1018 /* see if we have it already opened */
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	1019 for (file = log->files; file != NULL; file = file->next) {
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1020 if (file->st_ino == st.st_ino &&
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1021 CMP_DEV_T(file->st_dev, st.st_dev)) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1022 if (close(fd) < 0)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1023 i_error("close() failed: %m");
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1024 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1025 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1026 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1027
5013 6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	1028
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	1029 file = mail_transaction_log_file_alloc(log, path);
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	1030 file->fd = fd;
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	1031
6c76ed030b60 Fixed a race condition in transaction log rotation, which could have caused Timo Sirainen <tss@iki.fi> parents: 4968 diff changeset	1032 ret = mail_transaction_log_file_fd_open(file, FALSE, TRUE);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1033 if (ret <= 0) {
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1034 bool stale = errno == ESTALE;
1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1035
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1036 if (ret == 0) {
3368 1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1037 /* corrupted, delete it */
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1038 if (unlink(file->filepath) < 0 && errno != ENOENT) {
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1039 i_error("unlink(%s) failed: %m",
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1040 file->filepath);
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1041 }
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1042 mail_transaction_log_file_free(file);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1043 return 0;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1044 }
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1045 mail_transaction_log_file_free(file);
1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1046
1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1047 if (stale) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1048 /* treat as "doesn't exist" */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1049 return 0;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1050 }
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1051 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1052 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1053
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1054 /* got it */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1055 mail_transaction_log_file_add_to_list(file);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1056
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1057 /* but is it what we expected? */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1058 if (file->hdr.file_seq != file_seq)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1059 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1060
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1061 *file_r = file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1062 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1063 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1064
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1065 static int
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1066 mail_transaction_log_file_sync(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1067 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1068 const struct mail_transaction_header *hdr;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1069 const void *data;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1070 size_t size;
3368 1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1071 uint32_t hdr_size = 0;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1072
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1073 data = buffer_get_data(file->buffer, &size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1074
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1075 if (file->sync_offset < file->buffer_offset)
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1076 file->sync_offset = file->buffer_offset;
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1077
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1078 while (file->sync_offset - file->buffer_offset + sizeof(*hdr) <= size) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1079 hdr = CONST_PTR_OFFSET(data, file->sync_offset -
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1080 file->buffer_offset);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1081 hdr_size = mail_index_offset_to_uint32(hdr->size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1082 if (hdr_size == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1083 /* unfinished */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1084 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1085 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1086 if (hdr_size < sizeof(*hdr)) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1087 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1088 "hdr.size too small (%u)", hdr_size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1089 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1090 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1091
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1092 if (file->sync_offset - file->buffer_offset + hdr_size > size)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1093 break;
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1094 file->sync_offset += hdr_size;
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1095 }
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1096
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1097 if (file->sync_offset - file->buffer_offset != size) {
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1098 /* record goes outside the file we've seen. or if
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1099 we're accessing the log file via unlocked mmaped
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1100 memory, it may be just that the memory was updated
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1101 after we checked the file size. */
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1102 if (file->locked \|\| file->mmap_base == NULL) {
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1103 mail_transaction_log_file_set_corrupted(file,
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1104 "hdr.size too large (%u)", hdr_size);
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1105 return -1;
3218 746c309e366c If transaction log record size is too large, mark the file corrupted. Timo Sirainen <tss@iki.fi> parents: 3180 diff changeset	1106 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1107 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1108 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1109 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1110
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1111 static int
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1112 mail_transaction_log_file_read(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1113 uoff_t offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1114 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1115 void *data;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1116 size_t size;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1117 uint32_t read_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1118 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1119
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1120 i_assert(file->mmap_base == NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1121
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1122 if (file->buffer != NULL && file->buffer_offset > offset) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1123 /* we have to insert missing data to beginning of buffer */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1124 size = file->buffer_offset - offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1125 buffer_copy(file->buffer, size, file->buffer, 0, (size_t)-1);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1126 file->buffer_offset -= size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1127
1932 f04ff2a2ad5d fix Timo Sirainen <tss@iki.fi> parents: 1927 diff changeset	1128 data = buffer_get_space_unsafe(file->buffer, 0, size);
1933 bfdff8c75965 fixes for mmap_disable Timo Sirainen <tss@iki.fi> parents: 1932 diff changeset	1129 ret = pread_full(file->fd, data, size, offset);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1130 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1131 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1132 "Unexpected end of file");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1133 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1134 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1135 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1136 if (errno == ESTALE) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1137 /* log file was deleted in NFS server,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1138 fail silently */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1139 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1140 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1141 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1142 file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1143 "pread()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1144 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1145 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1146 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1147
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1148 if (file->buffer == NULL) {
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1149 file->buffer =
f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1150 buffer_create_dynamic(default_pool, LOG_PREFETCH);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1151 file->buffer_offset = offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1152 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1153
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1154 /* read all records */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1155 read_offset = file->buffer_offset + buffer_get_used_size(file->buffer);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1156
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1157 do {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1158 data = buffer_append_space_unsafe(file->buffer, LOG_PREFETCH);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1159 ret = pread(file->fd, data, LOG_PREFETCH, read_offset);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1160 if (ret > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1161 read_offset += ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1162
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1163 size = read_offset - file->buffer_offset;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1164 buffer_set_used_size(file->buffer, size);
2746 bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1165 } while (ret > 0 \|\| (ret < 0 && errno == EINTR));
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1166
2746 bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1167 if (mail_transaction_log_file_sync(file) < 0)
bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1168 return -1;
1919 4aeb7357951a locking fixes, mmap_disable=yes fixes Timo Sirainen <tss@iki.fi> parents: 1917 diff changeset	1169
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1170 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1171 /* EOF */
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1172 i_assert(file->sync_offset >= file->buffer_offset);
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1173 buffer_set_used_size(file->buffer,
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1174 file->sync_offset - file->buffer_offset);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1175 return 1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1176 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1177
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1178 if (errno == ESTALE) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1179 /* log file was deleted in NFS server, fail silently */
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1180 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1181 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1182
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1183 mail_index_file_set_syscall_error(file->log->index, file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1184 "pread()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1185 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1186 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1187
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1188 int mail_transaction_log_file_map(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1189 uoff_t start_offset, uoff_t end_offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1190 {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1191 struct mail_index *index = file->log->index;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1192 size_t size;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1193 struct stat st;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1194 int ret, use_mmap;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1195
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1196 i_assert(start_offset <= end_offset);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1197
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1198 if (file->hdr.indexid == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1199 /* corrupted */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1200 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1201 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1202
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	1203 if (start_offset < file->hdr.hdr_size) {
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1204 mail_transaction_log_file_set_corrupted(file,
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1205 "offset (%"PRIuUOFF_T") < header size (%u)",
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	1206 start_offset, file->hdr.hdr_size);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1207 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1208 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1209
5017 987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1210 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1211 return 1;
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1212
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1213 /* with mmap_no_write we could alternatively just write to log with
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1214 msync() rather than pwrite(). but since there aren't many such OSes
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1215 left, it's easier to just use mmap_disable behavior with it */
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1216 use_mmap = !index->mmap_disable && !index->mmap_no_write;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1217
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1218 if (file->buffer != NULL && file->buffer_offset <= start_offset) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1219 /* see if we already have it */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1220 size = buffer_get_used_size(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1221 if (file->buffer_offset + size >= end_offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1222 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1223 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1224
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1225 if (use_mmap) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1226 if (fstat(file->fd, &st) < 0) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1227 mail_index_file_set_syscall_error(index, file->filepath,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1228 "fstat()");
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1229 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1230 }
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1231 if (start_offset > (uoff_t)st.st_size) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1232 mail_transaction_log_file_set_corrupted(file,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1233 "start_offset (%"PRIuUOFF_T") > file size "
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1234 "(%"PRIuUOFF_T")", start_offset,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1235 (uoff_t)st.st_size);
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1236 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1237 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1238
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1239 if (file->mmap_base != NULL &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1240 (uoff_t)st.st_size == file->mmap_size &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1241 file->buffer_offset <= start_offset &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1242 end_offset == (uoff_t)-1) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1243 /* it's all mmaped already */
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1244 if (mail_transaction_log_file_sync(file) < 0)
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1245 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1246 return 1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1247 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1248 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1249
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1250 if (file->buffer != NULL &&
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1251 (file->mmap_base != NULL \|\| use_mmap)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1252 buffer_free(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1253 file->buffer = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1254 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1255 if (file->mmap_base != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1256 if (munmap(file->mmap_base, file->mmap_size) < 0) {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1257 mail_index_file_set_syscall_error(index, file->filepath,
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1258 "munmap()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1259 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1260 file->mmap_base = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1261 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1262
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1263 if (!use_mmap) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1264 ret = mail_transaction_log_file_read(file, start_offset);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1265 if (ret <= 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1266 /* make sure we don't leave ourself in
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1267 inconsistent state */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1268 if (file->buffer != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1269 buffer_free(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1270 file->buffer = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1271 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1272 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1273 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1274 } else {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1275 file->mmap_size = st.st_size;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1276 file->mmap_base = mmap(NULL, file->mmap_size, PROT_READ,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1277 MAP_SHARED, file->fd, 0);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1278 if (file->mmap_base == MAP_FAILED) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1279 file->mmap_base = NULL;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1280 mail_index_file_set_syscall_error(index, file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1281 "mmap()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1282 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1283 }
2440 0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1284
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1285 if (file->mmap_size > mmap_get_page_size()) {
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1286 if (madvise(file->mmap_base, file->mmap_size,
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1287 MADV_SEQUENTIAL) < 0) {
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1288 mail_index_file_set_syscall_error(index,
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1289 file->filepath, "madvise()");
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1290 }
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1291 }
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1292
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1293 file->buffer = buffer_create_const_data(default_pool,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1294 file->mmap_base,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1295 file->mmap_size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1296 file->buffer_offset = 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1297
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1298 if (mail_transaction_log_file_sync(file) < 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1299 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1300 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1301
5017 987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1302 if (start_offset > file->sync_offset) {
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1303 mail_transaction_log_file_set_corrupted(file,
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1304 "start_offset (%"PRIuUOFF_T") > current sync_offset "
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1305 "(%"PRIuUOFF_T")", start_offset, file->sync_offset);
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1306 return -1;
987397c331d7 Fixes to error handling paths. Timo Sirainen <tss@iki.fi> parents: 5015 diff changeset	1307 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1308 if (end_offset != (uoff_t)-1 && end_offset > file->sync_offset) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1309 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1310 "end_offset (%"PRIuUOFF_T") > current sync_offset "
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1311 "(%"PRIuUOFF_T")", end_offset, file->sync_offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1312 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1313 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1314
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1315 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1316 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1317
3146 9a8bf1c86699 Moved code into mail-transaction-log-append.c Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	1318 int mail_transaction_log_lock_head(struct mail_transaction_log *log)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1319 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1320 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1321 int ret = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1322
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1323 /* we want to get the head file locked. this is a bit racy,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1324 since by the time we have it locked a new log file may have been
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1325 created.
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1326
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1327 creating new log file requires locking the head file, so if we
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1328 can lock it and don't see another file, we can be sure no-one is
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1329 creating a new log at the moment */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1330
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1331 for (;;) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1332 file = log->head;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1333 if (mail_transaction_log_file_lock(file) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1334 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1335
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1336 file->refcount++;
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	1337 ret = mail_transaction_log_refresh(log, TRUE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1338 if (--file->refcount == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1339 mail_transaction_logs_clean(log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1340 file = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1341 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1342
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1343 if (ret == 0 && log->head == file) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1344 /* success */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1345 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1346 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1347
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1348 if (file != NULL)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1349 mail_transaction_log_file_unlock(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1350
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1351 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1352 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1353
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1354 /* try again */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1355 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1356
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1357 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1358 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1359
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1360 int mail_transaction_log_sync_lock(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1361 uint32_t file_seq_r, uoff_t file_offset_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1362 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1363 i_assert(!log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1364
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1365 if (mail_transaction_log_lock_head(log) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1366 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1367
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1368 /* update sync_offset */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1369 if (mail_transaction_log_file_map(log->head, log->head->sync_offset,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1370 (uoff_t)-1) < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1371 mail_transaction_log_file_unlock(log->head);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1372 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1373 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1374
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1375 log->index->log_locked = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1376 *file_seq_r = log->head->hdr.file_seq;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1377 *file_offset_r = log->head->sync_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1378 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1379 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1380
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1381 void mail_transaction_log_sync_unlock(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1382 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1383 i_assert(log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1384
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1385 log->index->log_locked = FALSE;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1386 mail_transaction_log_file_unlock(log->head);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1387 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1388
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1389 void mail_transaction_log_get_head(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1390 uint32_t file_seq_r, uoff_t file_offset_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1391 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1392 i_assert(log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1393
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1394 *file_seq_r = log->head->hdr.file_seq;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1395 *file_offset_r = log->head->sync_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1396 }
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1397
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1398 bool mail_transaction_log_is_head_prev(struct mail_transaction_log *log,
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1399 uint32_t file_seq, uoff_t file_offset)
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1400 {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1401 return log->head->hdr.prev_file_seq == file_seq &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1402 log->head->hdr.prev_file_offset == file_offset;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1403 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-transaction-log.c @ 5236:6761b2a46ec3 HEAD