dovecot/original-hg/dovecot-1.2: src/lib-index/mail-transaction-log.c annotate

annotate src/lib-index/mail-transaction-log.c @ 4785:a18fa2b4bf06 HEAD

Code cleanup

author	Timo Sirainen <tss@iki.fi>
date	Sun, 12 Nov 2006 13:15:38 +0200
parents	1dc7ffa3c669
children	7cc1b8fec8ab

rev	line source
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1 /* Copyright (C) 2003-2004 Timo Sirainen */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
1976 4e0269adb24a Don't rotate transaction log too often Timo Sirainen <tss@iki.fi> parents: 1974 diff changeset	4 #include "ioloop.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "buffer.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	6 #include "file-dotlock.h"
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4023 diff changeset	7 #include "nfs-workarounds.h"
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	8 #include "close-keep-errno.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	9 #include "read-full.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	10 #include "write-full.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	11 #include "mmap-util.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	12 #include "mail-index-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	13 #include "mail-index-view-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	14 #include "mail-transaction-log-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	15 #include "mail-transaction-util.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	16 #include "mail-index-transaction-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	17
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	18 #include <stddef.h>
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	19 #include <stdio.h>
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	20 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	21
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	22 #define LOG_PREFETCH 1024
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	23
1920 d11efbf1d341 Added fcntl_lock_disable setting to allow indexes to work with NFS. Some Timo Sirainen <tss@iki.fi> parents: 1919 diff changeset	24 /* this lock should never exist for a long time.. */
3350 2af6d96e4134 Updated timeouts Timo Sirainen <tss@iki.fi> parents: 3342 diff changeset	25 #define LOG_DOTLOCK_TIMEOUT 60
3981 0d64f8888dcd Removed immediate_stale_timeout and changed the stale_timeout behavior to Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	26 #define LOG_DOTLOCK_STALE_TIMEOUT 60
1920 d11efbf1d341 Added fcntl_lock_disable setting to allow indexes to work with NFS. Some Timo Sirainen <tss@iki.fi> parents: 1919 diff changeset	27
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	28 #define MAIL_TRANSACTION_LOG_SUFFIX ".log"
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	29 #define LOG_NEW_DOTLOCK_SUFFIX ".newlock"
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	30
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	31 static struct mail_transaction_log_file *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	32 mail_transaction_log_file_open_or_create(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	33 const char *path);
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	34 static int
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	35 mail_transaction_log_file_read(struct mail_transaction_log_file *file,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	36 uoff_t offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	37
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	38 void
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	39 mail_transaction_log_file_set_corrupted(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	40 const char *fmt, ...)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	41 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	42 va_list va;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	43
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	44 file->hdr.indexid = 0;
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	45 if (!MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	46 if (pwrite_full(file->fd, &file->hdr.indexid,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	47 sizeof(file->hdr.indexid), 0) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	48 mail_index_file_set_syscall_error(file->log->index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	49 file->filepath, "pwrite()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	50 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	51 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	52
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	53 va_start(va, fmt);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	54 t_push();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	55 mail_index_set_error(file->log->index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	56 "Corrupted transaction log file %s: %s",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	57 file->filepath, t_strdup_vprintf(fmt, va));
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 t_pop();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	59 va_end(va);
2862 6a8c73bfbcad fsck index if transaction log is found to be corrupted. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	60
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	61 if (file->log->index->log != NULL) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	62 /* this may have happened because of broken index.
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	63 make sure it's ok. */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	64 (void)mail_index_fsck(file->log->index);
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	65 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	66 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	67
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	68 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	69 mail_transaction_log_file_dotlock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	70 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	71 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	72
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	73 if (file->log->dotlock_count > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	74 ret = 1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	75 else {
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	76 ret = file_dotlock_create(&file->log->dotlock_settings,
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	77 file->filepath, 0,
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	78 &file->log->dotlock);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	79 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	80 if (ret > 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	81 file->log->dotlock_count++;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	82 file->locked = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	83 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	84 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	85 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	86 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	87 file->filepath,
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	88 "file_dotlock_create()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	89 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	90 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	91
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	92 mail_index_set_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	93 "Timeout while waiting for release of "
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	94 "dotlock for transaction log file %s",
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	95 file->filepath);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	96 file->log->index->index_lock_timeout = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	97 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	98 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	99
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	100 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	101 mail_transaction_log_file_undotlock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	102 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	103 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	104
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	105 if (--file->log->dotlock_count > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	106 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	107
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	108 ret = file_dotlock_delete(&file->log->dotlock);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	109 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	110 mail_index_file_set_syscall_error(file->log->index,
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	111 file->filepath, "file_dotlock_delete()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	112 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	113 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	114
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	115 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	116 mail_index_set_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	117 "Dotlock was lost for transaction log file %s",
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	118 file->filepath);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	119 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	120 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	121 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	122 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	123
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	124 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	125 mail_transaction_log_file_lock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	126 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	127 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	128
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	129 if (file->locked)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	130 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	131
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	132 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	133 file->locked = TRUE;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	134 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	135 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	136
2725 ccd07e584938 typofixes Timo Sirainen <tss@iki.fi> parents: 2723 diff changeset	137 if (file->log->index->lock_method == MAIL_INDEX_LOCK_DOTLOCK)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	138 return mail_transaction_log_file_dotlock(file);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	139
2941 89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	140 ret = mail_index_lock_fd(file->log->index, file->filepath, file->fd,
89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	141 F_WRLCK, MAIL_INDEX_LOCK_SECS);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	142 if (ret > 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	143 file->locked = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	144 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	145 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	146 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	147 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	148 file->filepath,
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	149 "mail_index_wait_lock_fd()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	150 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	151 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	152
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	153 mail_index_set_error(file->log->index,
3552 6a1ea27a4256 More error message fixing Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	154 "Timeout while waiting for lock for transaction log file %s",
6a1ea27a4256 More error message fixing Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	155 file->filepath);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	156 file->log->index->index_lock_timeout = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	157 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	158 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	159
3146 9a8bf1c86699 Moved code into mail-transaction-log-append.c Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	160 void mail_transaction_log_file_unlock(struct mail_transaction_log_file *file)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	161 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	162 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	163
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	164 if (!file->locked)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	165 return;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	166
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	167 file->locked = FALSE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	168
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	169 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	170 return;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	171
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	172 if (file->log->index->lock_method == MAIL_INDEX_LOCK_DOTLOCK) {
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	173 mail_transaction_log_file_undotlock(file);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	174 return;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	175 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	176
2941 89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	177 ret = mail_index_lock_fd(file->log->index, file->filepath, file->fd,
89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	178 F_UNLCK, 0);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	179 if (ret <= 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	180 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	181 file->filepath,
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	182 "mail_index_wait_lock_fd()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	183 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	184 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	185
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	186 #define INDEX_HAS_MISSING_LOGS(index, file) \
1952 05f76fe32f0d Don't crash in assert() with corrupted index file. Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	187 !(((file)->hdr.file_seq == (index)->hdr->log_file_seq && \
3170 42e1e9dbf19e hdr_size fix Timo Sirainen <tss@iki.fi> parents: 3169 diff changeset	188 (index)->hdr->log_file_int_offset >= (file)->hdr.hdr_size) \|\| \
1952 05f76fe32f0d Don't crash in assert() with corrupted index file. Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	189 ((file)->hdr.prev_file_seq == (index)->hdr->log_file_seq && \
2819 bea9c88c8763 Use separate sync offsets for internal/external transactions. Pending external Timo Sirainen <tss@iki.fi> parents: 2815 diff changeset	190 (file)->hdr.prev_file_offset == (index)->hdr->log_file_int_offset))
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	191
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	192 static int mail_transaction_log_check_file_seq(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	193 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	194 struct mail_index *index = log->index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	195 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	196 unsigned int lock_id;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	197 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	198
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	199 if (mail_transaction_log_lock_head(log) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	200 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	201
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	202 file = log->head;
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	203 file->refcount++;
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	204
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	205 ret = mail_index_lock_shared(index, TRUE, &lock_id);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	206 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	207 ret = mail_index_map(index, FALSE);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	208 if (ret <= 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	209 ret = -1;
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	210 else if (INDEX_HAS_MISSING_LOGS(index, file)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	211 /* broken - fix it by creating a new log file */
2815 ed85b5a62e06 fix Timo Sirainen <tss@iki.fi> parents: 2762 diff changeset	212 ret = mail_transaction_log_rotate(log, FALSE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	213 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	214 }
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	215
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	216 if (--file->refcount == 0)
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	217 mail_transaction_logs_clean(log);
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	218 else
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	219 mail_transaction_log_file_unlock(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	220 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	221 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	222
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	223 struct mail_transaction_log *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	224 mail_transaction_log_open_or_create(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	225 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	226 struct mail_transaction_log *log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	227 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	228
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	229 log = i_new(struct mail_transaction_log, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	230 log->index = index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	231
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	232 log->dotlock_settings.timeout = LOG_DOTLOCK_TIMEOUT;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	233 log->dotlock_settings.stale_timeout = LOG_DOTLOCK_STALE_TIMEOUT;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	234
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	235 log->new_dotlock_settings = log->dotlock_settings;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	236 log->new_dotlock_settings.lock_suffix = LOG_NEW_DOTLOCK_SUFFIX;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	237
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	238 path = t_strconcat(log->index->filepath,
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	239 MAIL_TRANSACTION_LOG_SUFFIX, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	240 log->head = mail_transaction_log_file_open_or_create(log, path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	241 if (log->head == NULL) {
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	242 /* fallback to in-memory indexes */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	243 if (mail_index_move_to_memory(index) < 0) {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	244 mail_transaction_log_close(&log);
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	245 return NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	246 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	247 log->head = mail_transaction_log_file_open_or_create(log, path);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	248 i_assert(log->head != NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	249 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	250
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	251 if (index->fd != -1 &&
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	252 INDEX_HAS_MISSING_LOGS(index, log->head)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	253 /* head log file isn't same as head index file -
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	254 shouldn't happen except in race conditions.
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	255 lock them and check again */
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	256 if (mail_transaction_log_check_file_seq(log) < 0) {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	257 mail_transaction_log_close(&log);
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	258 return NULL;
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	259 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	260 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	261 return log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	262 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	263
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	264 void mail_transaction_log_close(struct mail_transaction_log **_log)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	265 {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	266 struct mail_transaction_log log = _log;
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	267
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1941 diff changeset	268 mail_transaction_log_views_close(log);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	269
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	270 if (log->head != NULL)
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	271 log->head->refcount--;
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	272 mail_transaction_logs_clean(log);
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	273 i_assert(log->files == NULL);
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	274
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	275 *_log = NULL;
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1941 diff changeset	276 log->index->log = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	277 i_free(log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	278 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	279
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	280 static void
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	281 mail_transaction_log_file_free(struct mail_transaction_log_file *file)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	282 {
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	283 struct mail_transaction_log_file **p;
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	284 int old_errno = errno;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	285
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	286 mail_transaction_log_file_unlock(file);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	287
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	288 for (p = &file->log->files; p != NULL; p = &(p)->next) {
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	289 if (*p == file) {
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	290 *p = file->next;
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	291 break;
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	292 }
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	293 }
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	294
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	295 if (file == file->log->head)
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	296 file->log->head = NULL;
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	297
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	298 if (file->buffer != NULL)
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	299 buffer_free(file->buffer);
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	300
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	301 if (file->mmap_base != NULL) {
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	302 if (munmap(file->mmap_base, file->mmap_size) < 0) {
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	303 mail_index_file_set_syscall_error(file->log->index,
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	304 file->filepath,
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	305 "munmap()");
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	306 }
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	307 }
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	308
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	309 if (file->fd != -1) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	310 if (close(file->fd) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	311 mail_index_file_set_syscall_error(file->log->index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	312 file->filepath,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	313 "close()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	314 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	315 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	316
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	317 i_free(file->filepath);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	318 i_free(file);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	319
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	320 errno = old_errno;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	321 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	322
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	323 int mail_transaction_log_move_to_memory(struct mail_transaction_log *log)
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	324 {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	325 struct mail_transaction_log_file *file = log->head;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	326
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	327 if (file == NULL \|\| MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	328 return 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	329
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	330 /* read the whole file to memory. we might currently be appending
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	331 data into it, so we want to read it up to end of file */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	332 file->buffer_offset = 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	333
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	334 if (file->buffer != NULL) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	335 buffer_free(file->buffer);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	336 file->buffer = NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	337 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	338
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	339 if (file->mmap_base != NULL) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	340 if (munmap(file->mmap_base, file->mmap_size) < 0) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	341 mail_index_file_set_syscall_error(file->log->index,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	342 file->filepath,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	343 "munmap()");
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	344 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	345 file->mmap_base = NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	346 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	347
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	348 if (mail_transaction_log_file_read(file, 0) <= 0)
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	349 return -1;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	350
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	351 /* after we've read the file into memory, make it into in-memory
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	352 log file */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	353 if (close(file->fd) < 0) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	354 mail_index_file_set_syscall_error(file->log->index,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	355 file->filepath, "close()");
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	356 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	357 file->fd = -1;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	358 return 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	359 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	360
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	361 static int
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	362 mail_transaction_log_file_read_hdr(struct mail_transaction_log_file *file,
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	363 int head, bool ignore_estale)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	364 {
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	365 struct mail_transaction_log_file *f;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	366 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	367
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	368 i_assert(!MAIL_INDEX_IS_IN_MEMORY(file->log->index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	369
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	370 ret = pread_full(file->fd, &file->hdr, sizeof(file->hdr), 0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	371 if (ret < 0) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	372 if (errno != ESTALE \|\| !ignore_estale) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	373 mail_index_file_set_syscall_error(file->log->index,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	374 file->filepath,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	375 "pread_full()");
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	376 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	377 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	378 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	379 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	380 mail_transaction_log_file_set_corrupted(file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	381 "unexpected end of file while reading header");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	382 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	383 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	384
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	385 if (file->hdr.major_version != MAIL_TRANSACTION_LOG_MAJOR_VERSION) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	386 /* incompatible version - fix silently */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	387 return 0;
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	388 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	389 if (file->hdr.hdr_size < MAIL_TRANSACTION_LOG_HEADER_MIN_SIZE) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	390 mail_transaction_log_file_set_corrupted(file,
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	391 "Header size too small");
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	392 return 0;
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	393 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	394 if (file->hdr.hdr_size < sizeof(file->hdr)) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	395 /* @UNSAFE: smaller than we expected - zero out the fields we
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	396 shouldn't have filled */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	397 memset(PTR_OFFSET(&file->hdr, file->hdr.hdr_size), 0,
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	398 sizeof(file->hdr) - file->hdr.hdr_size);
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	399 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	400
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	401 if (file->hdr.indexid == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	402 /* corrupted */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	403 mail_index_set_error(file->log->index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	404 "Transaction log file %s: marked corrupted",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	405 file->filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	406 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	407 }
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	408 if (file->hdr.indexid != file->log->index->indexid) {
3556 6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	409 if (file->log->index->fd != -1) {
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	410 /* index file was probably just rebuilt and we don't
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	411 know about it yet */
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	412 mail_transaction_log_file_set_corrupted(file,
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	413 "invalid indexid (%u != %u)",
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	414 file->hdr.indexid, file->log->index->indexid);
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	415 return 0;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	416 }
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	417
3556 6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	418 /* creating index file. since transaction log is created
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	419 first, use the indexid in it to create the main index
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	420 to avoid races. */
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	421 file->log->index->indexid = file->hdr.indexid;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	422 }
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	423
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	424 /* make sure we already don't have a file with the same sequence
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	425 opened. it shouldn't happen unless the old log file was
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	426 corrupted.
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	427
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	428 If we're opening head log file, make sure the sequence is larger
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	429 than any existing one. */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	430 if (head) {
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	431 for (f = file->log->files; f != NULL; f = f->next) {
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	432 if (f->hdr.file_seq >= file->hdr.file_seq) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	433 mail_transaction_log_file_set_corrupted(file,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	434 "invalid new transaction log sequence "
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	435 "(%u >= %u)",
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	436 f->hdr.file_seq, file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	437 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	438 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	439 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	440 } else {
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	441 for (f = file->log->files; f != NULL; f = f->next) {
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	442 if (f->hdr.file_seq == file->hdr.file_seq) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	443 mail_transaction_log_file_set_corrupted(file,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	444 "old transaction log already opened "
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	445 "(%u == %u)",
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	446 f->hdr.file_seq, file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	447 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	448 }
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	449 }
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	450 }
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	451
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	452 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	453 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	454
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	455 static int
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	456 mail_transaction_log_init_hdr(struct mail_transaction_log *log,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	457 struct mail_transaction_log_header *hdr)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	458 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	459 struct mail_index *index = log->index;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	460 unsigned int lock_id;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	461
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	462 memset(hdr, 0, sizeof(*hdr));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	463 hdr->major_version = MAIL_TRANSACTION_LOG_MAJOR_VERSION;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	464 hdr->minor_version = MAIL_TRANSACTION_LOG_MINOR_VERSION;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	465 hdr->hdr_size = sizeof(struct mail_transaction_log_header);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	466 hdr->indexid = log->index->indexid;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	467 hdr->create_stamp = ioloop_time;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	468
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	469 if (index->fd != -1) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	470 /* not creating index - make sure we have latest header */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	471 if (mail_index_lock_shared(index, TRUE, &lock_id) < 0)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	472 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	473 if (mail_index_map(index, FALSE) <= 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	474 mail_index_unlock(index, lock_id);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	475 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	476 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	477 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	478 hdr->prev_file_seq = index->hdr->log_file_seq;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	479 hdr->prev_file_offset = index->hdr->log_file_int_offset;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	480 hdr->file_seq = index->hdr->log_file_seq+1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	481
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	482 if (index->fd != -1)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	483 mail_index_unlock(index, lock_id);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	484
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	485 if (log->head != NULL && hdr->file_seq <= log->head->hdr.file_seq) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	486 /* make sure the sequence grows */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	487 hdr->file_seq = log->head->hdr.file_seq+1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	488 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	489 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	490 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	491
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	492 static int
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	493 mail_transaction_log_file_create2(struct mail_transaction_log *log,
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	494 const char *path, int new_fd,
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	495 struct dotlock **dotlock,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	496 dev_t dev, ino_t ino, uoff_t file_size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	497 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	498 struct mail_index *index = log->index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	499 struct mail_transaction_log_header hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	500 struct stat st;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	501 const char *path2;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	502 int old_fd, ret;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	503 bool found;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	504
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	505 /* log creation is locked now - see if someone already created it */
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4023 diff changeset	506 old_fd = nfs_safe_open(path, O_RDWR);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	507 if (old_fd != -1) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	508 if ((ret = fstat(old_fd, &st)) < 0) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	509 mail_index_file_set_syscall_error(index, path,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	510 "fstat()");
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	511 } else if (st.st_ino == ino && CMP_DEV_T(st.st_dev, dev) &&
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	512 (uoff_t)st.st_size == file_size) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	513 /* same file, still broken */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	514 } else {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	515 /* file changed, use the new file */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	516 (void)file_dotlock_delete(dotlock);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	517 return old_fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	518 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	519
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	520 (void)close(old_fd);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	521 old_fd = -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	522
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	523 if (ret < 0) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	524 /* fstat() failure, return after closing fd.. */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	525 return -1;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	526 }
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	527 found = TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	528 } else if (errno != ENOENT) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	529 mail_index_file_set_syscall_error(index, path, "open()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	530 return -1;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	531 } else {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	532 found = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	533 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	534
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	535 if (mail_transaction_log_init_hdr(log, &hdr) < 0)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	536 return -1;
1936 bb1bde2680b0 assert Timo Sirainen <tss@iki.fi> parents: 1935 diff changeset	537
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	538 if (write_full(new_fd, &hdr, sizeof(hdr)) < 0) {
1974 d85f71ffeb8f file_dotlock_open/delete/replace now has lock_suffix parameter. NULL Timo Sirainen <tss@iki.fi> parents: 1967 diff changeset	539 mail_index_file_set_syscall_error(index, path,
d85f71ffeb8f file_dotlock_open/delete/replace now has lock_suffix parameter. NULL Timo Sirainen <tss@iki.fi> parents: 1967 diff changeset	540 "write_full()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	541 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	542 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	543
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	544 /* keep two log files */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	545 if (found) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	546 path2 = t_strconcat(path, ".2", NULL);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	547 if (rename(path, path2) < 0) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	548 mail_index_set_error(index,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	549 "rename(%s, %s) failed: %m", path, path2);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	550 /* ignore the error. we don't care that much about the
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	551 second log file and we're going to overwrite this
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	552 first one. */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	553 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	554 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	555
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	556 if (file_dotlock_replace(dotlock,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	557 DOTLOCK_REPLACE_FLAG_DONT_CLOSE_FD) <= 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	558 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	559
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	560 /* success */
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	561 return new_fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	562 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	563
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	564 static int
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	565 mail_transaction_log_file_create(struct mail_transaction_log *log,
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	566 const char *path,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	567 dev_t dev, ino_t ino, uoff_t file_size)
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	568 {
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	569 struct dotlock *dotlock;
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	570 struct stat st;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	571 mode_t old_mask;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	572 int fd;
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	573
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	574 i_assert(!MAIL_INDEX_IS_IN_MEMORY(log->index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	575
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	576 if (stat(log->index->dir, &st) < 0) {
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	577 if (ENOTFOUND(errno)) {
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	578 /* the whole index directory was deleted, which means
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	579 the mailbox was deleted by another process.
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	580 fail silently. */
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	581 mail_index_mark_corrupted(log->index);
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	582 return -1;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	583 }
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	584 mail_index_file_set_syscall_error(log->index, log->index->dir,
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	585 "stat()");
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	586 return -1;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	587 }
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	588
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	589 /* With dotlocking we might already have path.lock created, so this
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	590 filename has to be different. */
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	591 old_mask = umask(log->index->mode ^ 0666);
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	592 fd = file_dotlock_open(&log->new_dotlock_settings, path, 0, &dotlock);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	593 umask(old_mask);
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	594
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	595 if (fd == -1) {
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	596 mail_index_file_set_syscall_error(log->index, path,
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	597 "file_dotlock_open()");
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	598 return -1;
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	599 }
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	600
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	601 if (log->index->gid != (gid_t)-1 &&
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	602 fchown(fd, (uid_t)-1, log->index->gid) < 0) {
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	603 mail_index_file_set_syscall_error(log->index, path, "fchown()");
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	604 (void)file_dotlock_delete(&dotlock);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	605 return -1;
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	606 }
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	607
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	608 /* either fd gets used or the dotlock gets deleted and returned fd
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	609 is for the existing file */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	610 fd = mail_transaction_log_file_create2(log, path, fd, &dotlock,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	611 dev, ino, file_size);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	612 if (fd < 0) {
3317 a19268b76093 Don't crash if dotlock replacing failed. Timo Sirainen <tss@iki.fi> parents: 3316 diff changeset	613 if (dotlock != NULL)
a19268b76093 Don't crash if dotlock replacing failed. Timo Sirainen <tss@iki.fi> parents: 3316 diff changeset	614 (void)file_dotlock_delete(&dotlock);
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	615 return -1;
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	616 }
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	617 return fd;
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	618 }
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	619
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	620 static void
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	621 mail_transaction_log_file_add_to_head(struct mail_transaction_log_file *file)
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	622 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	623 struct mail_transaction_log *log = file->log;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	624 struct mail_transaction_log_file **p;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	625
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	626 if (log->index->map != NULL &&
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	627 file->hdr.file_seq == log->index->map->hdr.log_file_seq &&
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	628 log->index->map->hdr.log_file_int_offset != 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	629 /* we can get a valid log offset from index file. initialize
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	630 sync_offset from it so we don't have to read the whole log
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	631 file from beginning. */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	632 file->sync_offset = log->index->map->hdr.log_file_int_offset;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	633 } else {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	634 file->sync_offset = file->hdr.hdr_size;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	635 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	636
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	637 /* append to end of list. */
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	638 for (p = &log->files; p != NULL; p = &(p)->next)
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	639 i_assert((*p)->hdr.file_seq < file->hdr.file_seq);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	640 *p = file;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	641 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	642
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	643 static void
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	644 mail_transaction_log_file_add_to_list(struct mail_transaction_log_file *file)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	645 {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	646 struct mail_transaction_log *log = file->log;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	647 struct mail_transaction_log_file **p;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	648
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	649 file->sync_offset = file->hdr.hdr_size;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	650
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	651 /* insert it to correct position */
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	652 for (p = &log->files; p != NULL; p = &(p)->next) {
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	653 i_assert((*p)->hdr.file_seq != file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	654 if ((*p)->hdr.file_seq > file->hdr.file_seq)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	655 break;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	656 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	657
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	658 file->next = *p;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	659 *p = file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	660 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	661
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	662 static int
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	663 mail_transaction_log_file_fd_open(struct mail_transaction_log *log,
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	664 struct mail_transaction_log_file **file_r,
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	665 const char *path, int fd, bool head,
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	666 bool ignore_estale)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	667 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	668 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	669 struct stat st;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	670 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	671
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	672 i_assert(!MAIL_INDEX_IS_IN_MEMORY(log->index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	673
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	674 *file_r = NULL;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	675
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	676 if (fstat(fd, &st) < 0) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	677 if (errno != ESTALE \|\| !ignore_estale) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	678 mail_index_file_set_syscall_error(log->index, path,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	679 "fstat()");
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	680 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	681 close_keep_errno(fd);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	682 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	683 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	684
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	685 file = i_new(struct mail_transaction_log_file, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	686 file->refcount = 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	687 file->log = log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	688 file->filepath = i_strdup(path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	689 file->fd = fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	690 file->st_dev = st.st_dev;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	691 file->st_ino = st.st_ino;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	692 file->last_mtime = st.st_mtime;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	693 file->last_size = st.st_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	694
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	695 ret = mail_transaction_log_file_read_hdr(file, head, ignore_estale);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	696 if (ret < 0) {
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	697 mail_transaction_log_file_free(file);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	698 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	699 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	700
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	701 *file_r = file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	702 return ret;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	703 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	704
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	705
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	706 static struct mail_transaction_log_file *
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	707 mail_transaction_log_file_fd_open_or_create(struct mail_transaction_log *log,
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	708 const char *path, int fd,
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	709 bool *retry_r, bool try_retry)
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	710 {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	711 struct mail_transaction_log_file *file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	712 struct stat st;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	713 int ret;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	714
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	715 *retry_r = FALSE;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	716
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	717 ret = mail_transaction_log_file_fd_open(log, &file, path, fd, TRUE,
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	718 !try_retry);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	719 if (ret < 0) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	720 *retry_r = errno == ESTALE && try_retry;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	721 return NULL;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	722 }
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	723
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	724 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	725 /* corrupted header */
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	726 fd = mail_transaction_log_file_create(log, path, file->st_dev,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	727 file->st_ino,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	728 file->last_size);
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	729 if (fd == -1)
2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	730 ret = -1;
2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	731 else if (fstat(fd, &st) < 0) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	732 mail_index_file_set_syscall_error(log->index, path,
2013 c4ac66804f8a fix Timo Sirainen <tss@iki.fi> parents: 2009 diff changeset	733 "fstat()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	734 (void)close(fd);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	735 fd = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	736 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	737 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	738
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	739 if (fd != -1) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	740 (void)close(file->fd);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	741 file->fd = fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	742
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	743 file->st_dev = st.st_dev;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	744 file->st_ino = st.st_ino;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	745 file->last_mtime = st.st_mtime;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	746 file->last_size = st.st_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	747
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	748 memset(&file->hdr, 0, sizeof(file->hdr));
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	749 ret = mail_transaction_log_file_read_hdr(file, TRUE,
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	750 !try_retry);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	751 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	752 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	753 if (ret <= 0) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	754 *retry_r = errno == ESTALE && try_retry;
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	755 mail_transaction_log_file_free(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	756 return NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	757 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	758
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	759 mail_transaction_log_file_add_to_head(file);
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	760 return file;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	761 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	762
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	763 static struct mail_transaction_log_file *
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	764 mail_transaction_log_file_alloc_in_memory(struct mail_transaction_log *log)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	765 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	766 struct mail_transaction_log_file *file;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	767
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	768 file = i_new(struct mail_transaction_log_file, 1);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	769 file->refcount = 1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	770 file->log = log;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	771 file->filepath = i_strdup("(in-memory transaction log file)");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	772 file->fd = -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	773
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	774 if (mail_transaction_log_init_hdr(log, &file->hdr) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	775 i_free(file);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	776 return NULL;
2449 1543762353aa Optimization for setting initial sync_offset from index. Timo Sirainen <tss@iki.fi> parents: 2440 diff changeset	777 }
1543762353aa Optimization for setting initial sync_offset from index. Timo Sirainen <tss@iki.fi> parents: 2440 diff changeset	778
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	779 file->buffer = buffer_create_dynamic(default_pool, 4096);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	780 file->buffer_offset = sizeof(file->hdr);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	781
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	782 mail_transaction_log_file_add_to_head(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	783 return file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	784 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	785
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	786 static struct mail_transaction_log_file *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	787 mail_transaction_log_file_open_or_create(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	788 const char *path)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	789 {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	790 struct mail_transaction_log_file *file;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	791 unsigned int i;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	792 bool retry;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	793 int fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	794
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	795 if (MAIL_INDEX_IS_IN_MEMORY(log->index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	796 return mail_transaction_log_file_alloc_in_memory(log);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	797
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	798 for (i = 0; ; i++) {
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4023 diff changeset	799 fd = nfs_safe_open(path, O_RDWR);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	800 if (fd == -1) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	801 if (errno != ENOENT) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	802 mail_index_file_set_syscall_error(log->index,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	803 path,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	804 "open()");
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	805 return NULL;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	806 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	807
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	808 /* doesn't exist, try creating it */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	809 fd = mail_transaction_log_file_create(log, path,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	810 0, 0, 0);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	811 if (fd == -1)
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	812 return NULL;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	813 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	814
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	815 file = mail_transaction_log_file_fd_open_or_create(log, path,
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	816 fd, &retry, i == MAIL_INDEX_ESTALE_RETRY_COUNT);
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	817 if (file != NULL \|\| !retry)
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	818 return file;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	819
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	820 /* ESTALE - retry */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	821 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	822 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	823
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	824 static struct mail_transaction_log_file *
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	825 mail_transaction_log_file_open(struct mail_transaction_log *log,
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	826 const char *path)
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	827 {
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	828 struct mail_transaction_log_file *file;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	829 unsigned int i;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	830 int fd, ret;
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	831
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	832 for (i = 0;; i++) {
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4023 diff changeset	833 fd = nfs_safe_open(path, O_RDWR);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	834 if (fd == -1) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	835 mail_index_file_set_syscall_error(log->index, path,
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	836 "open()");
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	837 return NULL;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	838 }
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	839
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	840 ret = mail_transaction_log_file_fd_open(log, &file, path,
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	841 fd, TRUE, i < MAIL_INDEX_ESTALE_RETRY_COUNT);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	842 if (ret > 0)
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	843 break;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	844
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	845 /* error / corrupted */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	846 if (ret == 0)
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	847 mail_transaction_log_file_free(file);
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	848 else {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	849 if (errno == ESTALE &&
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	850 i < MAIL_INDEX_ESTALE_RETRY_COUNT) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	851 /* try again */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	852 continue;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	853 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	854 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	855 return NULL;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	856 }
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	857
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	858 mail_transaction_log_file_add_to_head(file);
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	859 return file;
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	860 }
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	861
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	862 void mail_transaction_logs_clean(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	863 {
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	864 struct mail_transaction_log_file file, next;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	865
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	866 for (file = log->files; file != NULL; file = next) {
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	867 next = file->next;
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	868
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	869 if (file->refcount == 0)
a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	870 mail_transaction_log_file_free(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	871 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	872 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	873
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	874 int mail_transaction_log_rotate(struct mail_transaction_log *log, bool lock)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	875 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	876 struct mail_transaction_log_file *file;
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	877 const char *path = log->head->filepath;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	878 struct stat st;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	879 bool retry;
2008 23959140f4f5 some minor fixes Timo Sirainen <tss@iki.fi> parents: 2004 diff changeset	880 int fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	881
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	882 i_assert(log->head->locked);
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	883
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	884 if (MAIL_INDEX_IS_IN_MEMORY(log->index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	885 file = mail_transaction_log_file_alloc_in_memory(log);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	886 else {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	887 /* we're locked, we shouldn't need to worry about ESTALE
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	888 problems in here. */
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	889 if (fstat(log->head->fd, &st) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	890 mail_index_file_set_syscall_error(log->index, path,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	891 "fstat()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	892 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	893 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	894
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	895 fd = mail_transaction_log_file_create(log, path,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	896 st.st_dev, st.st_ino,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	897 st.st_size);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	898 if (fd == -1)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	899 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	900
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	901 file = mail_transaction_log_file_fd_open_or_create(log, path,
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	902 fd, &retry, FALSE);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	903 if (file == NULL) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	904 i_assert(!retry);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	905 return -1;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	906 }
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	907 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	908
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	909 if (lock) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	910 if (mail_transaction_log_file_lock(file) < 0) {
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	911 file->refcount--;
2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	912 mail_transaction_logs_clean(log);
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	913 return -1;
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	914 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	915 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	916 i_assert(file->locked == lock);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	917
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	918 if (--log->head->refcount == 0)
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	919 mail_transaction_logs_clean(log);
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	920 else
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	921 mail_transaction_log_file_unlock(log->head);
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	922
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	923 i_assert(log->head != file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	924 log->head = file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	925 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	926 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	927
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	928 static int mail_transaction_log_refresh(struct mail_transaction_log *log,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	929 bool create_if_needed)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	930 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	931 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	932 struct stat st;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	933 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	934
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	935 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(log->head))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	936 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	937
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	938 path = t_strconcat(log->index->filepath,
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	939 MAIL_TRANSACTION_LOG_SUFFIX, NULL);
4225 abdef9d86a94 Added nfs_safe_stat() to transparently work around ESTALE with stat(). Timo Sirainen <timo.sirainen@movial.fi> parents: 4072 diff changeset	940 if (nfs_safe_stat(path, &st) < 0) {
4396 208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	941 if (errno != ENOENT) {
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	942 mail_index_file_set_syscall_error(log->index, path,
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	943 "stat()");
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	944 return -1;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	945 }
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	946 /* log was deleted. just reopen/recreate it. */
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	947 } else {
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	948 if (log->head != NULL &&
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	949 log->head->st_ino == st.st_ino &&
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	950 CMP_DEV_T(log->head->st_dev, st.st_dev)) {
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	951 /* same file */
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	952 return 0;
208194b3742a Handle transaction log deletions silently. Handle whole index directory Timo Sirainen <tss@iki.fi> parents: 4225 diff changeset	953 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	954 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	955
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	956 file = create_if_needed ?
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	957 mail_transaction_log_file_open_or_create(log, path) :
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	958 mail_transaction_log_file_open(log, path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	959 if (file == NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	960 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	961
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	962 i_assert(!file->locked);
2095 92d8b70829c8 fixes Timo Sirainen <tss@iki.fi> parents: 2073 diff changeset	963
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	964 if (log->head != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	965 if (--log->head->refcount == 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	966 mail_transaction_logs_clean(log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	967 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	968
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	969 log->head = file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	970 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	971 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	972
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	973 int mail_transaction_log_file_find(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	974 uint32_t file_seq,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	975 struct mail_transaction_log_file **file_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	976 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	977 struct mail_transaction_log_file *file;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	978 struct stat st;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	979 const char *path;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	980 int ret, fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	981
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	982 if (file_seq > log->head->hdr.file_seq) {
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	983 /* don't try to recreate log file if it gets lost. we're
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	984 already in trouble and with mmap_disable the creation
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	985 could cause a recursive mail_index_map() call */
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	986 if (mail_transaction_log_refresh(log, FALSE) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	987 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	988 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	989
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	990 for (file = log->files; file != NULL; file = file->next) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	991 if (file->hdr.file_seq == file_seq) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	992 *file_r = file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	993 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	994 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	995 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	996
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	997 /* see if we have it in log.2 file */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	998 path = t_strconcat(log->index->filepath,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	999 MAIL_TRANSACTION_LOG_SUFFIX".2", NULL);
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4023 diff changeset	1000 fd = nfs_safe_open(path, O_RDWR);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1001 if (fd == -1) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1002 if (errno == ENOENT)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1003 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1004
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1005 mail_index_file_set_syscall_error(log->index, path, "open()");
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1006 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1007 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1008
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1009 if (fstat(fd, &st) < 0) {
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1010 close_keep_errno(fd);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1011 if (errno == ESTALE) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1012 /* treat as "doesn't exist" */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1013 return 0;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1014 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1015 mail_index_file_set_syscall_error(log->index, path, "fstat()");
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1016 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1017 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1018
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1019 /* see if we have it already opened */
4785 a18fa2b4bf06 Code cleanup Timo Sirainen <tss@iki.fi> parents: 4777 diff changeset	1020 for (file = log->files; file != NULL; file = file->next) {
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1021 if (file->st_ino == st.st_ino &&
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1022 CMP_DEV_T(file->st_dev, st.st_dev)) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1023 if (close(fd) < 0)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1024 i_error("close() failed: %m");
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1025 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1026 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1027 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1028
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1029 ret = mail_transaction_log_file_fd_open(log, &file, path, fd,
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	1030 FALSE, TRUE);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1031 if (ret <= 0) {
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1032 bool stale = errno == ESTALE;
1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1033
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1034 if (ret == 0) {
3368 1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1035 /* corrupted, delete it */
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1036 if (unlink(file->filepath) < 0 && errno != ENOENT) {
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1037 i_error("unlink(%s) failed: %m",
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1038 file->filepath);
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1039 }
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	1040 mail_transaction_log_file_free(file);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1041 return 0;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1042 }
4777 1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1043 mail_transaction_log_file_free(file);
1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1044
1dc7ffa3c669 Don't leak opened .log.2 transaction logs. Timo Sirainen <tss@iki.fi> parents: 4396 diff changeset	1045 if (stale) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1046 /* treat as "doesn't exist" */
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1047 return 0;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	1048 }
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1049 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1050 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1051
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1052 /* got it */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1053 mail_transaction_log_file_add_to_list(file);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1054
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1055 /* but is it what we expected? */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1056 if (file->hdr.file_seq != file_seq)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1057 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1058
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1059 *file_r = file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	1060 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1061 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1062
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1063 static int
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1064 mail_transaction_log_file_sync(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1065 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1066 const struct mail_transaction_header *hdr;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1067 const void *data;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1068 size_t size;
3368 1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	1069 uint32_t hdr_size = 0;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1070
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1071 data = buffer_get_data(file->buffer, &size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1072
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1073 if (file->sync_offset < file->buffer_offset)
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1074 file->sync_offset = file->buffer_offset;
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	1075
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1076 while (file->sync_offset - file->buffer_offset + sizeof(*hdr) <= size) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1077 hdr = CONST_PTR_OFFSET(data, file->sync_offset -
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1078 file->buffer_offset);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1079 hdr_size = mail_index_offset_to_uint32(hdr->size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1080 if (hdr_size == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1081 /* unfinished */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1082 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1083 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1084 if (hdr_size < sizeof(*hdr)) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1085 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1086 "hdr.size too small (%u)", hdr_size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1087 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1088 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1089
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1090 if (file->sync_offset - file->buffer_offset + hdr_size > size)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1091 break;
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1092 file->sync_offset += hdr_size;
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1093 }
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1094
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1095 if (file->sync_offset - file->buffer_offset != size) {
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1096 /* record goes outside the file we've seen. or if
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1097 we're accessing the log file via unlocked mmaped
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1098 memory, it may be just that the memory was updated
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1099 after we checked the file size. */
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1100 if (file->locked \|\| file->mmap_base == NULL) {
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1101 mail_transaction_log_file_set_corrupted(file,
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1102 "hdr.size too large (%u)", hdr_size);
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1103 return -1;
3218 746c309e366c If transaction log record size is too large, mark the file corrupted. Timo Sirainen <tss@iki.fi> parents: 3180 diff changeset	1104 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1105 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1106 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1107 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1108
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1109 static int
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1110 mail_transaction_log_file_read(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1111 uoff_t offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1112 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1113 void *data;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1114 size_t size;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1115 uint32_t read_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1116 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1117
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1118 i_assert(file->mmap_base == NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1119
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1120 if (file->buffer != NULL && file->buffer_offset > offset) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1121 /* we have to insert missing data to beginning of buffer */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1122 size = file->buffer_offset - offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1123 buffer_copy(file->buffer, size, file->buffer, 0, (size_t)-1);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1124 file->buffer_offset -= size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1125
1932 f04ff2a2ad5d fix Timo Sirainen <tss@iki.fi> parents: 1927 diff changeset	1126 data = buffer_get_space_unsafe(file->buffer, 0, size);
1933 bfdff8c75965 fixes for mmap_disable Timo Sirainen <tss@iki.fi> parents: 1932 diff changeset	1127 ret = pread_full(file->fd, data, size, offset);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1128 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1129 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1130 "Unexpected end of file");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1131 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1132 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1133 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1134 if (errno == ESTALE) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1135 /* log file was deleted in NFS server,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1136 fail silently */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1137 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1138 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1139 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1140 file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1141 "pread()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1142 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1143 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1144 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1145
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1146 if (file->buffer == NULL) {
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1147 file->buffer =
f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1148 buffer_create_dynamic(default_pool, LOG_PREFETCH);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1149 file->buffer_offset = offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1150 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1151
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1152 /* read all records */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1153 read_offset = file->buffer_offset + buffer_get_used_size(file->buffer);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1154
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1155 do {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1156 data = buffer_append_space_unsafe(file->buffer, LOG_PREFETCH);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1157 ret = pread(file->fd, data, LOG_PREFETCH, read_offset);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1158 if (ret > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1159 read_offset += ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1160
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1161 size = read_offset - file->buffer_offset;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1162 buffer_set_used_size(file->buffer, size);
2746 bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1163 } while (ret > 0 \|\| (ret < 0 && errno == EINTR));
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1164
2746 bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1165 if (mail_transaction_log_file_sync(file) < 0)
bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1166 return -1;
1919 4aeb7357951a locking fixes, mmap_disable=yes fixes Timo Sirainen <tss@iki.fi> parents: 1917 diff changeset	1167
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1168 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1169 /* EOF */
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1170 i_assert(file->sync_offset >= file->buffer_offset);
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1171 buffer_set_used_size(file->buffer,
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1172 file->sync_offset - file->buffer_offset);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1173 return 1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1174 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1175
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1176 if (errno == ESTALE) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1177 /* log file was deleted in NFS server, fail silently */
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1178 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1179 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1180
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1181 mail_index_file_set_syscall_error(file->log->index, file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1182 "pread()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1183 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1184 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1185
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1186 int mail_transaction_log_file_map(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1187 uoff_t start_offset, uoff_t end_offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1188 {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1189 struct mail_index *index = file->log->index;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1190 size_t size;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1191 struct stat st;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1192 int ret, use_mmap;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1193
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1194 i_assert(start_offset <= end_offset);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1195
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1196 if (file->hdr.indexid == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1197 /* corrupted */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1198 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1199 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1200
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	1201 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	1202 return 1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	1203
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	1204 if (start_offset < file->hdr.hdr_size) {
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1205 mail_transaction_log_file_set_corrupted(file,
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1206 "offset (%"PRIuUOFF_T") < header size (%u)",
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	1207 start_offset, file->hdr.hdr_size);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1208 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1209 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1210
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1211 /* with mmap_no_write we could alternatively just write to log with
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1212 msync() rather than pwrite(). but since there aren't many such OSes
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1213 left, it's easier to just use mmap_disable behavior with it */
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1214 use_mmap = !index->mmap_disable && !index->mmap_no_write;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1215
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1216 if (file->buffer != NULL && file->buffer_offset <= start_offset) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1217 /* see if we already have it */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1218 size = buffer_get_used_size(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1219 if (file->buffer_offset + size >= end_offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1220 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1221 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1222
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1223 if (use_mmap) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1224 if (fstat(file->fd, &st) < 0) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1225 mail_index_file_set_syscall_error(index, file->filepath,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1226 "fstat()");
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1227 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1228 }
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1229 if (start_offset > (uoff_t)st.st_size) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1230 mail_transaction_log_file_set_corrupted(file,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1231 "start_offset (%"PRIuUOFF_T") > file size "
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1232 "(%"PRIuUOFF_T")", start_offset,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1233 (uoff_t)st.st_size);
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1234 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1235 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1236
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1237 if (file->mmap_base != NULL &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1238 (uoff_t)st.st_size == file->mmap_size &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1239 file->buffer_offset <= start_offset &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1240 end_offset == (uoff_t)-1) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1241 /* it's all mmaped already */
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1242 if (mail_transaction_log_file_sync(file) < 0)
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1243 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1244 return 1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1245 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1246 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1247
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1248 if (file->buffer != NULL &&
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1249 (file->mmap_base != NULL \|\| use_mmap)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1250 buffer_free(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1251 file->buffer = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1252 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1253 if (file->mmap_base != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1254 if (munmap(file->mmap_base, file->mmap_size) < 0) {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1255 mail_index_file_set_syscall_error(index, file->filepath,
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1256 "munmap()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1257 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1258 file->mmap_base = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1259 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1260
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1261 if (!use_mmap) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1262 ret = mail_transaction_log_file_read(file, start_offset);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1263 if (ret <= 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1264 /* make sure we don't leave ourself in
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1265 inconsistent state */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1266 if (file->buffer != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1267 buffer_free(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1268 file->buffer = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1269 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1270 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1271 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1272 } else {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1273 file->mmap_size = st.st_size;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1274 file->mmap_base = mmap(NULL, file->mmap_size, PROT_READ,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1275 MAP_SHARED, file->fd, 0);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1276 if (file->mmap_base == MAP_FAILED) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1277 file->mmap_base = NULL;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1278 mail_index_file_set_syscall_error(index, file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1279 "mmap()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1280 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1281 }
2440 0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1282
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1283 if (file->mmap_size > mmap_get_page_size()) {
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1284 if (madvise(file->mmap_base, file->mmap_size,
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1285 MADV_SEQUENTIAL) < 0) {
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1286 mail_index_file_set_syscall_error(index,
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1287 file->filepath, "madvise()");
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1288 }
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1289 }
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1290
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1291 file->buffer = buffer_create_const_data(default_pool,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1292 file->mmap_base,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1293 file->mmap_size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1294 file->buffer_offset = 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1295
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1296 if (mail_transaction_log_file_sync(file) < 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1297 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1298 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1299
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1300 if (end_offset != (uoff_t)-1 && end_offset > file->sync_offset) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1301 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1302 "end_offset (%"PRIuUOFF_T") > current sync_offset "
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1303 "(%"PRIuUOFF_T")", end_offset, file->sync_offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1304 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1305 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1306
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1307 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1308 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1309
3146 9a8bf1c86699 Moved code into mail-transaction-log-append.c Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	1310 int mail_transaction_log_lock_head(struct mail_transaction_log *log)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1311 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1312 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1313 int ret = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1314
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1315 /* we want to get the head file locked. this is a bit racy,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1316 since by the time we have it locked a new log file may have been
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1317 created.
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1318
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1319 creating new log file requires locking the head file, so if we
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1320 can lock it and don't see another file, we can be sure no-one is
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1321 creating a new log at the moment */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1322
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1323 for (;;) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1324 file = log->head;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1325 if (mail_transaction_log_file_lock(file) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1326 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1327
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1328 file->refcount++;
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	1329 ret = mail_transaction_log_refresh(log, TRUE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1330 if (--file->refcount == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1331 mail_transaction_logs_clean(log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1332 file = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1333 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1334
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1335 if (ret == 0 && log->head == file) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1336 /* success */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1337 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1338 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1339
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1340 if (file != NULL)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1341 mail_transaction_log_file_unlock(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1342
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1343 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1344 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1345
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1346 /* try again */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1347 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1348
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1349 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1350 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1351
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1352 int mail_transaction_log_sync_lock(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1353 uint32_t file_seq_r, uoff_t file_offset_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1354 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1355 i_assert(!log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1356
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1357 if (mail_transaction_log_lock_head(log) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1358 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1359
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1360 /* update sync_offset */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1361 if (mail_transaction_log_file_map(log->head, log->head->sync_offset,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1362 (uoff_t)-1) < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1363 mail_transaction_log_file_unlock(log->head);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1364 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1365 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1366
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1367 log->index->log_locked = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1368 *file_seq_r = log->head->hdr.file_seq;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1369 *file_offset_r = log->head->sync_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1370 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1371 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1372
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1373 void mail_transaction_log_sync_unlock(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1374 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1375 i_assert(log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1376
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1377 log->index->log_locked = FALSE;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1378 mail_transaction_log_file_unlock(log->head);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1379 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1380
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1381 void mail_transaction_log_get_head(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1382 uint32_t file_seq_r, uoff_t file_offset_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1383 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1384 i_assert(log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1385
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1386 *file_seq_r = log->head->hdr.file_seq;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1387 *file_offset_r = log->head->sync_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1388 }
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1389
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1390 bool mail_transaction_log_is_head_prev(struct mail_transaction_log *log,
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1391 uint32_t file_seq, uoff_t file_offset)
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1392 {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1393 return log->head->hdr.prev_file_seq == file_seq &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1394 log->head->hdr.prev_file_offset == file_offset;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1395 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-transaction-log.c @ 4785:a18fa2b4bf06 HEAD