dovecot/original-hg/dovecot-1.2: src/lib-index/mail-transaction-log.c annotate

annotate src/lib-index/mail-transaction-log.c @ 3879:928229f8b3e6 HEAD

deinit, unref, destroy, close, free, etc. functions now take a pointer to their data pointer, and set it to NULL. This makes double-frees less likely to cause security holes.

author	Timo Sirainen <tss@iki.fi>
date	Sat, 14 Jan 2006 20:47:20 +0200
parents	55df57c028d4
children	0d64f8888dcd

rev	line source
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1 /* Copyright (C) 2003-2004 Timo Sirainen */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 #include "lib.h"
1976 4e0269adb24a Don't rotate transaction log too often Timo Sirainen <tss@iki.fi> parents: 1974 diff changeset	4 #include "ioloop.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	5 #include "buffer.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	6 #include "file-dotlock.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	7 #include "read-full.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	8 #include "write-full.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	9 #include "mmap-util.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	10 #include "mail-index-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	11 #include "mail-index-view-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	12 #include "mail-transaction-log-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	13 #include "mail-transaction-util.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	14 #include "mail-index-transaction-private.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	15
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	16 #include <stddef.h>
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	17 #include <stdio.h>
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	18 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	19
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	20 #define LOG_PREFETCH 1024
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	21
1920 d11efbf1d341 Added fcntl_lock_disable setting to allow indexes to work with NFS. Some Timo Sirainen <tss@iki.fi> parents: 1919 diff changeset	22 /* this lock should never exist for a long time.. */
3350 2af6d96e4134 Updated timeouts Timo Sirainen <tss@iki.fi> parents: 3342 diff changeset	23 #define LOG_DOTLOCK_TIMEOUT 60
1920 d11efbf1d341 Added fcntl_lock_disable setting to allow indexes to work with NFS. Some Timo Sirainen <tss@iki.fi> parents: 1919 diff changeset	24 #define LOG_DOTLOCK_STALE_TIMEOUT 0
3350 2af6d96e4134 Updated timeouts Timo Sirainen <tss@iki.fi> parents: 3342 diff changeset	25 #define LOG_DOTLOCK_IMMEDIATE_STALE_TIMEOUT 60
1920 d11efbf1d341 Added fcntl_lock_disable setting to allow indexes to work with NFS. Some Timo Sirainen <tss@iki.fi> parents: 1919 diff changeset	26
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	27 #define MAIL_TRANSACTION_LOG_SUFFIX ".log"
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	28 #define LOG_NEW_DOTLOCK_SUFFIX ".newlock"
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	29
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	30 static struct mail_transaction_log_file *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	31 mail_transaction_log_file_open_or_create(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	32 const char *path);
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	33 static int
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	34 mail_transaction_log_file_read(struct mail_transaction_log_file *file,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	35 uoff_t offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	36
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	37 void
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	38 mail_transaction_log_file_set_corrupted(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	39 const char *fmt, ...)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	40 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	41 va_list va;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	42
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	43 file->hdr.indexid = 0;
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	44 if (!MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	45 if (pwrite_full(file->fd, &file->hdr.indexid,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	46 sizeof(file->hdr.indexid), 0) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	47 mail_index_file_set_syscall_error(file->log->index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	48 file->filepath, "pwrite()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	49 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	50 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	51
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	52 va_start(va, fmt);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	53 t_push();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	54 mail_index_set_error(file->log->index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	55 "Corrupted transaction log file %s: %s",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	56 file->filepath, t_strdup_vprintf(fmt, va));
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	57 t_pop();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 va_end(va);
2862 6a8c73bfbcad fsck index if transaction log is found to be corrupted. Timo Sirainen <tss@iki.fi> parents: 2853 diff changeset	59
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	60 if (file->log->index->log != NULL) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	61 /* this may have happened because of broken index.
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	62 make sure it's ok. */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	63 (void)mail_index_fsck(file->log->index);
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	64 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	65 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	66
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	67 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	68 mail_transaction_log_file_dotlock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	69 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	70 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	71
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	72 if (file->log->dotlock_count > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	73 ret = 1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	74 else {
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	75 ret = file_dotlock_create(&file->log->dotlock_settings,
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	76 file->filepath, 0,
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	77 &file->log->dotlock);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	78 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	79 if (ret > 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	80 file->log->dotlock_count++;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	81 file->locked = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	82 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	83 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	84 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	85 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	86 file->filepath,
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	87 "file_dotlock_create()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	88 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	89 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	90
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	91 mail_index_set_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	92 "Timeout while waiting for release of "
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	93 "dotlock for transaction log file %s",
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	94 file->filepath);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	95 file->log->index->index_lock_timeout = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	96 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	97 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	98
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	99 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	100 mail_transaction_log_file_undotlock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	101 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	102 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	103
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	104 if (--file->log->dotlock_count > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	105 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	106
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	107 ret = file_dotlock_delete(&file->log->dotlock);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	108 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	109 mail_index_file_set_syscall_error(file->log->index,
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	110 file->filepath, "file_dotlock_delete()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	111 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	112 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	113
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	114 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	115 mail_index_set_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	116 "Dotlock was lost for transaction log file %s",
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	117 file->filepath);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	118 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	119 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	120 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	121 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	122
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	123 static int
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	124 mail_transaction_log_file_lock(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	125 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	126 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	127
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	128 if (file->locked)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	129 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	130
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	131 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file)) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	132 file->locked = TRUE;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	133 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	134 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	135
2725 ccd07e584938 typofixes Timo Sirainen <tss@iki.fi> parents: 2723 diff changeset	136 if (file->log->index->lock_method == MAIL_INDEX_LOCK_DOTLOCK)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	137 return mail_transaction_log_file_dotlock(file);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	138
2941 89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	139 ret = mail_index_lock_fd(file->log->index, file->filepath, file->fd,
89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	140 F_WRLCK, MAIL_INDEX_LOCK_SECS);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	141 if (ret > 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	142 file->locked = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	143 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	144 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	145 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	146 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	147 file->filepath,
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	148 "mail_index_wait_lock_fd()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	149 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	150 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	151
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	152 mail_index_set_error(file->log->index,
3552 6a1ea27a4256 More error message fixing Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	153 "Timeout while waiting for lock for transaction log file %s",
6a1ea27a4256 More error message fixing Timo Sirainen <tss@iki.fi> parents: 3551 diff changeset	154 file->filepath);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	155 file->log->index->index_lock_timeout = TRUE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	156 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	157 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	158
3146 9a8bf1c86699 Moved code into mail-transaction-log-append.c Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	159 void mail_transaction_log_file_unlock(struct mail_transaction_log_file *file)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	160 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	161 int ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	162
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	163 if (!file->locked)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	164 return;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	165
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	166 file->locked = FALSE;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	167
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	168 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	169 return;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	170
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	171 if (file->log->index->lock_method == MAIL_INDEX_LOCK_DOTLOCK) {
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	172 mail_transaction_log_file_undotlock(file);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	173 return;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	174 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	175
2941 89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	176 ret = mail_index_lock_fd(file->log->index, file->filepath, file->fd,
89ab5c72430d lock_method=dotlock doesn't crash anymore while trying to modify cache file. Timo Sirainen <tss@iki.fi> parents: 2931 diff changeset	177 F_UNLCK, 0);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	178 if (ret <= 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	179 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	180 file->filepath,
2723 12b503fbb8af Replaced fcntl_locks_disable with lock_method, so it's now possible to use Timo Sirainen <tss@iki.fi> parents: 2708 diff changeset	181 "mail_index_wait_lock_fd()");
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	182 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	183 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	184
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	185 #define INDEX_HAS_MISSING_LOGS(index, file) \
1952 05f76fe32f0d Don't crash in assert() with corrupted index file. Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	186 !(((file)->hdr.file_seq == (index)->hdr->log_file_seq && \
3170 42e1e9dbf19e hdr_size fix Timo Sirainen <tss@iki.fi> parents: 3169 diff changeset	187 (index)->hdr->log_file_int_offset >= (file)->hdr.hdr_size) \|\| \
1952 05f76fe32f0d Don't crash in assert() with corrupted index file. Timo Sirainen <tss@iki.fi> parents: 1942 diff changeset	188 ((file)->hdr.prev_file_seq == (index)->hdr->log_file_seq && \
2819 bea9c88c8763 Use separate sync offsets for internal/external transactions. Pending external Timo Sirainen <tss@iki.fi> parents: 2815 diff changeset	189 (file)->hdr.prev_file_offset == (index)->hdr->log_file_int_offset))
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	190
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	191 static int mail_transaction_log_check_file_seq(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	192 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	193 struct mail_index *index = log->index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	194 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	195 unsigned int lock_id;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	196 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	197
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	198 if (mail_transaction_log_lock_head(log) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	199 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	200
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	201 file = log->head;
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	202 file->refcount++;
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	203
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	204 ret = mail_index_lock_shared(index, TRUE, &lock_id);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	205 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	206 ret = mail_index_map(index, FALSE);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	207 if (ret <= 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	208 ret = -1;
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	209 else if (INDEX_HAS_MISSING_LOGS(index, file)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	210 /* broken - fix it by creating a new log file */
2815 ed85b5a62e06 fix Timo Sirainen <tss@iki.fi> parents: 2762 diff changeset	211 ret = mail_transaction_log_rotate(log, FALSE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	212 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	213 }
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	214
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	215 if (--file->refcount == 0)
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	216 mail_transaction_logs_clean(log);
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	217 else
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	218 mail_transaction_log_file_unlock(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	219 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	220 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	221
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	222 struct mail_transaction_log *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	223 mail_transaction_log_open_or_create(struct mail_index *index)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	224 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	225 struct mail_transaction_log *log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	226 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	227
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	228 log = i_new(struct mail_transaction_log, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	229 log->index = index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	230
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	231 log->dotlock_settings.timeout = LOG_DOTLOCK_TIMEOUT;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	232 log->dotlock_settings.stale_timeout = LOG_DOTLOCK_STALE_TIMEOUT;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	233 log->dotlock_settings.immediate_stale_timeout =
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	234 LOG_DOTLOCK_IMMEDIATE_STALE_TIMEOUT;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	235
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	236 log->new_dotlock_settings = log->dotlock_settings;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	237 log->new_dotlock_settings.lock_suffix = LOG_NEW_DOTLOCK_SUFFIX;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	238
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	239 path = t_strconcat(log->index->filepath,
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	240 MAIL_TRANSACTION_LOG_SUFFIX, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	241 log->head = mail_transaction_log_file_open_or_create(log, path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	242 if (log->head == NULL) {
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	243 /* fallback to in-memory indexes */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	244 if (mail_index_move_to_memory(index) < 0) {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	245 mail_transaction_log_close(&log);
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	246 return NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	247 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	248 log->head = mail_transaction_log_file_open_or_create(log, path);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	249 i_assert(log->head != NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	250 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	251
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	252 if (index->fd != -1 &&
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	253 INDEX_HAS_MISSING_LOGS(index, log->head)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	254 /* head log file isn't same as head index file -
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	255 shouldn't happen except in race conditions.
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	256 lock them and check again */
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	257 if (mail_transaction_log_check_file_seq(log) < 0) {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	258 mail_transaction_log_close(&log);
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	259 return NULL;
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	260 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	261 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	262 return log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	263 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	264
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	265 void mail_transaction_log_close(struct mail_transaction_log **_log)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	266 {
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	267 struct mail_transaction_log log = _log;
928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	268
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1941 diff changeset	269 mail_transaction_log_views_close(log);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	270
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	271 if (log->head != NULL) {
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	272 log->head->refcount--;
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	273 mail_transaction_logs_clean(log);
9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 3090 diff changeset	274 }
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	275
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	276 *_log = NULL;
1942 2b114aa7eeec handle losing index file Timo Sirainen <tss@iki.fi> parents: 1941 diff changeset	277 log->index->log = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	278 i_free(log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	279 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	280
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	281 static void
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	282 mail_transaction_log_file_free(struct mail_transaction_log_file *file)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	283 {
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	284 mail_transaction_log_file_unlock(file);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	285
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	286 if (file == file->log->head)
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	287 file->log->head = NULL;
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	288 if (file == file->log->tail)
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	289 file->log->tail = file->next;
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	290
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	291 if (file->buffer != NULL)
1967 b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	292 buffer_free(file->buffer);
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	293
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	294 if (file->mmap_base != NULL) {
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	295 if (munmap(file->mmap_base, file->mmap_size) < 0) {
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	296 mail_index_file_set_syscall_error(file->log->index,
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	297 file->filepath,
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	298 "munmap()");
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	299 }
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	300 }
b6d900688c55 memory leak fixes Timo Sirainen <tss@iki.fi> parents: 1964 diff changeset	301
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	302 if (file->fd != -1) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	303 if (close(file->fd) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	304 mail_index_file_set_syscall_error(file->log->index,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	305 file->filepath,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	306 "close()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	307 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	308 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	309
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	310 i_free(file->filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	311 i_free(file);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	312 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	313
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	314 int mail_transaction_log_move_to_memory(struct mail_transaction_log *log)
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	315 {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	316 struct mail_transaction_log_file *file = log->head;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	317
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	318 if (file == NULL \|\| MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	319 return 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	320
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	321 /* read the whole file to memory. we might currently be appending
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	322 data into it, so we want to read it up to end of file */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	323 file->buffer_offset = 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	324
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	325 if (file->buffer != NULL) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	326 buffer_free(file->buffer);
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	327 file->buffer = NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	328 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	329
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	330 if (file->mmap_base != NULL) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	331 if (munmap(file->mmap_base, file->mmap_size) < 0) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	332 mail_index_file_set_syscall_error(file->log->index,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	333 file->filepath,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	334 "munmap()");
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	335 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	336 file->mmap_base = NULL;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	337 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	338
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	339 if (mail_transaction_log_file_read(file, 0) <= 0)
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	340 return -1;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	341
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	342 /* after we've read the file into memory, make it into in-memory
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	343 log file */
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	344 if (close(file->fd) < 0) {
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	345 mail_index_file_set_syscall_error(file->log->index,
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	346 file->filepath, "close()");
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	347 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	348 file->fd = -1;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	349 return 0;
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	350 }
318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	351
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	352 static int
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	353 mail_transaction_log_file_read_hdr(struct mail_transaction_log_file *file,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	354 int head)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	355 {
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	356 struct mail_transaction_log_file *f;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	357 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	358
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	359 i_assert(!MAIL_INDEX_IS_IN_MEMORY(file->log->index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	360
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	361 ret = pread_full(file->fd, &file->hdr, sizeof(file->hdr), 0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	362 if (ret < 0) {
1919 4aeb7357951a locking fixes, mmap_disable=yes fixes Timo Sirainen <tss@iki.fi> parents: 1917 diff changeset	363 // FIXME: handle ESTALE
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	364 mail_index_file_set_syscall_error(file->log->index,
1933 bfdff8c75965 fixes for mmap_disable Timo Sirainen <tss@iki.fi> parents: 1932 diff changeset	365 file->filepath,
bfdff8c75965 fixes for mmap_disable Timo Sirainen <tss@iki.fi> parents: 1932 diff changeset	366 "pread_full()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	367 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	368 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	369 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	370 mail_transaction_log_file_set_corrupted(file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	371 "unexpected end of file while reading header");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	372 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	373 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	374
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	375 if (file->hdr.major_version != MAIL_TRANSACTION_LOG_MAJOR_VERSION) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	376 /* incompatible version - fix silently */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	377 return 0;
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	378 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	379 if (file->hdr.hdr_size < MAIL_TRANSACTION_LOG_HEADER_MIN_SIZE) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	380 mail_transaction_log_file_set_corrupted(file,
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	381 "Header size too small");
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	382 return 0;
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	383 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	384 if (file->hdr.hdr_size < sizeof(file->hdr)) {
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	385 /* @UNSAFE: smaller than we expected - zero out the fields we
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	386 shouldn't have filled */
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	387 memset(PTR_OFFSET(&file->hdr, file->hdr.hdr_size), 0,
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	388 sizeof(file->hdr) - file->hdr.hdr_size);
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	389 }
ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	390
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	391 if (file->hdr.indexid == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	392 /* corrupted */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	393 mail_index_set_error(file->log->index,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	394 "Transaction log file %s: marked corrupted",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	395 file->filepath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	396 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	397 }
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	398 if (file->hdr.indexid != file->log->index->indexid) {
3556 6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	399 if (file->log->index->fd != -1) {
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	400 /* index file was probably just rebuilt and we don't
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	401 know about it yet */
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	402 mail_transaction_log_file_set_corrupted(file,
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	403 "invalid indexid (%u != %u)",
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	404 file->hdr.indexid, file->log->index->indexid);
2073 a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	405 return 0;
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	406 }
a58c51c8f37b Don't complain about transaction log indexid changes when rebuilding index. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	407
3556 6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	408 /* creating index file. since transaction log is created
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	409 first, use the indexid in it to create the main index
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	410 to avoid races. */
6170e2e5992a Index creation was racy. Timo Sirainen <tss@iki.fi> parents: 3555 diff changeset	411 file->log->index->indexid = file->hdr.indexid;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	412 }
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	413
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	414 /* make sure we already don't have a file with the same sequence
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	415 opened. it shouldn't happen unless the old log file was
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	416 corrupted.
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	417
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	418 If we're opening head log file, make sure the sequence is larger
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	419 than any existing one. */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	420 if (head) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	421 for (f = file->log->tail; f != NULL; f = f->next) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	422 if (f->hdr.file_seq >= file->hdr.file_seq) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	423 mail_transaction_log_file_set_corrupted(file,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	424 "invalid new transaction log sequence "
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	425 "(%u >= %u)",
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	426 f->hdr.file_seq, file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	427 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	428 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	429 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	430 } else {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	431 for (f = file->log->tail; f != NULL; f = f->next) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	432 if (f->hdr.file_seq == file->hdr.file_seq) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	433 mail_transaction_log_file_set_corrupted(file,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	434 "old transaction log already opened "
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	435 "(%u == %u)",
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	436 f->hdr.file_seq, file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	437 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	438 }
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	439 }
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	440 }
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	441
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	442 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	443 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	444
1935 ec4d5ff99f6e Don't modify index file when creating new transaction log. Timo Sirainen <tss@iki.fi> parents: 1933 diff changeset	445 static int
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	446 mail_transaction_log_init_hdr(struct mail_transaction_log *log,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	447 struct mail_transaction_log_header *hdr)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	448 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	449 struct mail_index *index = log->index;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	450 unsigned int lock_id;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	451
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	452 memset(hdr, 0, sizeof(*hdr));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	453 hdr->major_version = MAIL_TRANSACTION_LOG_MAJOR_VERSION;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	454 hdr->minor_version = MAIL_TRANSACTION_LOG_MINOR_VERSION;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	455 hdr->hdr_size = sizeof(struct mail_transaction_log_header);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	456 hdr->indexid = log->index->indexid;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	457 hdr->create_stamp = ioloop_time;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	458
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	459 if (index->fd != -1) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	460 /* not creating index - make sure we have latest header */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	461 if (mail_index_lock_shared(index, TRUE, &lock_id) < 0)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	462 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	463 if (mail_index_map(index, FALSE) <= 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	464 mail_index_unlock(index, lock_id);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	465 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	466 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	467 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	468 hdr->prev_file_seq = index->hdr->log_file_seq;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	469 hdr->prev_file_offset = index->hdr->log_file_int_offset;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	470 hdr->file_seq = index->hdr->log_file_seq+1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	471
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	472 if (index->fd != -1)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	473 mail_index_unlock(index, lock_id);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	474
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	475 if (log->head != NULL && hdr->file_seq <= log->head->hdr.file_seq) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	476 /* make sure the sequence grows */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	477 hdr->file_seq = log->head->hdr.file_seq+1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	478 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	479 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	480 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	481
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	482 static int
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	483 mail_transaction_log_file_create2(struct mail_transaction_log *log,
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	484 const char *path, int fd,
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	485 struct dotlock **dotlock,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	486 dev_t dev, ino_t ino, uoff_t file_size)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	487 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	488 struct mail_index *index = log->index;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	489 struct mail_transaction_log_header hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	490 struct stat st;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	491 const char *path2;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	492 int fd2, ret;
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	493 bool found;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	494
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	495 /* log creation is locked now - see if someone already created it */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	496 fd2 = open(path, O_RDWR);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	497 if (fd2 != -1) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	498 if ((ret = fstat(fd2, &st)) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	499 mail_index_file_set_syscall_error(index, path,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	500 "fstat()");
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	501 } else if (st.st_ino == ino && CMP_DEV_T(st.st_dev, dev) &&
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	502 (uoff_t)st.st_size == file_size) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	503 /* same file, still broken */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	504 } else {
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	505 (void)file_dotlock_delete(dotlock);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	506 return fd2;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	507 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	508
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	509 (void)close(fd2);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	510 fd2 = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	511
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	512 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	513 return -1;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	514 found = TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	515 } else if (errno != ENOENT) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	516 mail_index_file_set_syscall_error(index, path, "open()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	517 return -1;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	518 } else {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	519 found = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	520 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	521
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	522 if (mail_transaction_log_init_hdr(log, &hdr) < 0)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	523 return -1;
1936 bb1bde2680b0 assert Timo Sirainen <tss@iki.fi> parents: 1935 diff changeset	524
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	525 if (write_full(fd, &hdr, sizeof(hdr)) < 0) {
1974 d85f71ffeb8f file_dotlock_open/delete/replace now has lock_suffix parameter. NULL Timo Sirainen <tss@iki.fi> parents: 1967 diff changeset	526 mail_index_file_set_syscall_error(index, path,
d85f71ffeb8f file_dotlock_open/delete/replace now has lock_suffix parameter. NULL Timo Sirainen <tss@iki.fi> parents: 1967 diff changeset	527 "write_full()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	528 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	529 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	530
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	531 /* keep two log files */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	532 if (found) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	533 path2 = t_strconcat(path, ".2", NULL);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	534 if (rename(path, path2) < 0) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	535 i_error("rename(%s, %s) failed: %m", path, path2);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	536 /* ignore the error. we don't care that much about the
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	537 second log file and we're going to overwrite this
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	538 first one. */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	539 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	540 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	541
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	542 if (file_dotlock_replace(dotlock,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	543 DOTLOCK_REPLACE_FLAG_DONT_CLOSE_FD) <= 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	544 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	545
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	546 /* success */
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	547 return fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	548 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	549
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	550 static int
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	551 mail_transaction_log_file_create(struct mail_transaction_log *log,
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	552 const char *path,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	553 dev_t dev, ino_t ino, uoff_t file_size)
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	554 {
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	555 struct dotlock *dotlock;
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	556 mode_t old_mask;
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	557 int fd, fd2;
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	558
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	559 i_assert(!MAIL_INDEX_IS_IN_MEMORY(log->index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	560
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	561 /* With dotlocking we might already have path.lock created, so this
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	562 filename has to be different. */
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	563 old_mask = umask(log->index->mode ^ 0666);
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	564 fd = file_dotlock_open(&log->new_dotlock_settings, path, 0, &dotlock);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	565 umask(old_mask);
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	566
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	567 if (fd == -1) {
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	568 mail_index_file_set_syscall_error(log->index, path,
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	569 "file_dotlock_open()");
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	570 return -1;
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	571 }
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	572
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	573 if (log->index->gid != (gid_t)-1 &&
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	574 fchown(fd, (uid_t)-1, log->index->gid) < 0) {
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	575 mail_index_file_set_syscall_error(log->index, path, "fchown()");
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	576 (void)file_dotlock_delete(&dotlock);
2879 aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	577 return -1;
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	578 }
aa93c7216722 dovecot-shared file was supposed to show permissions/gid for files created Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	579
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	580 fd2 = mail_transaction_log_file_create2(log, path, fd, &dotlock,
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	581 dev, ino, file_size);
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	582 if (fd2 < 0) {
3317 a19268b76093 Don't crash if dotlock replacing failed. Timo Sirainen <tss@iki.fi> parents: 3316 diff changeset	583 if (dotlock != NULL)
a19268b76093 Don't crash if dotlock replacing failed. Timo Sirainen <tss@iki.fi> parents: 3316 diff changeset	584 (void)file_dotlock_delete(&dotlock);
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	585 return -1;
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	586 }
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	587 return fd2;
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	588 }
e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	589
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	590 static void
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	591 mail_transaction_log_file_add_to_head(struct mail_transaction_log_file *file)
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	592 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	593 struct mail_transaction_log *log = file->log;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	594 struct mail_transaction_log_file **p;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	595
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	596 if (log->index->map != NULL &&
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	597 file->hdr.file_seq == log->index->map->hdr.log_file_seq &&
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	598 log->index->map->hdr.log_file_int_offset != 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	599 /* we can get a valid log offset from index file. initialize
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	600 sync_offset from it so we don't have to read the whole log
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	601 file from beginning. */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	602 file->sync_offset = log->index->map->hdr.log_file_int_offset;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	603 } else {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	604 file->sync_offset = file->hdr.hdr_size;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	605 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	606
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	607 /* append to end of list. */
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	608 for (p = &log->tail; p != NULL; p = &(p)->next)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	609 i_assert((*p)->hdr.file_seq < file->hdr.file_seq);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	610 *p = file;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	611 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	612
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	613 static void
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	614 mail_transaction_log_file_add_to_list(struct mail_transaction_log_file *file)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	615 {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	616 struct mail_transaction_log *log = file->log;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	617 struct mail_transaction_log_file **p;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	618
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	619 file->sync_offset = file->hdr.hdr_size;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	620
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	621 /* insert it to correct position */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	622 for (p = &log->tail; p != NULL; p = &(p)->next) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	623 i_assert((*p)->hdr.file_seq != file->hdr.file_seq);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	624 if ((*p)->hdr.file_seq > file->hdr.file_seq)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	625 break;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	626 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	627
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	628 file->next = *p;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	629 *p = file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	630 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	631
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	632 static int
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	633 mail_transaction_log_file_fd_open(struct mail_transaction_log *log,
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	634 struct mail_transaction_log_file **file_r,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	635 const char *path, int fd, bool head)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	636 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	637 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	638 struct stat st;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	639 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	640
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	641 i_assert(!MAIL_INDEX_IS_IN_MEMORY(log->index));
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	642
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	643 *file_r = NULL;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	644
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	645 if (fstat(fd, &st) < 0) {
2013 c4ac66804f8a fix Timo Sirainen <tss@iki.fi> parents: 2009 diff changeset	646 mail_index_file_set_syscall_error(log->index, path, "fstat()");
2244 e7ccd299e754 fixed some fd leaks in error conditions Timo Sirainen <tss@iki.fi> parents: 2240 diff changeset	647 (void)close(fd);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	648 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	649 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	650
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	651 file = i_new(struct mail_transaction_log_file, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	652 file->refcount = 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	653 file->log = log;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	654 file->filepath = i_strdup(path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	655 file->fd = fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	656 file->st_dev = st.st_dev;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	657 file->st_ino = st.st_ino;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	658 file->last_mtime = st.st_mtime;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	659 file->last_size = st.st_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	660
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	661 ret = mail_transaction_log_file_read_hdr(file, head);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	662 if (ret < 0) {
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	663 mail_transaction_log_file_free(file);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	664 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	665 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	666
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	667 *file_r = file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	668 return ret;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	669 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	670
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	671
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	672 static struct mail_transaction_log_file *
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	673 mail_transaction_log_file_fd_open_or_create(struct mail_transaction_log *log,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	674 const char *path, int fd)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	675 {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	676 struct mail_transaction_log_file *file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	677 struct stat st;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	678 int ret;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	679
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	680 ret = mail_transaction_log_file_fd_open(log, &file, path, fd, TRUE);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	681 if (ret < 0)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	682 return NULL;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	683
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	684 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	685 /* corrupted header */
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	686 fd = mail_transaction_log_file_create(log, path, file->st_dev,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	687 file->st_ino,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	688 file->last_size);
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	689 if (fd == -1)
2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	690 ret = -1;
2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	691 else if (fstat(fd, &st) < 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	692 mail_index_file_set_syscall_error(log->index, path,
2013 c4ac66804f8a fix Timo Sirainen <tss@iki.fi> parents: 2009 diff changeset	693 "fstat()");
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	694 (void)close(fd);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	695 fd = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	696 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	697 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	698
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	699 if (fd != -1) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	700 (void)close(file->fd);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	701 file->fd = fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	702
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	703 file->st_dev = st.st_dev;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	704 file->st_ino = st.st_ino;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	705 file->last_mtime = st.st_mtime;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	706 file->last_size = st.st_size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	707
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	708 memset(&file->hdr, 0, sizeof(file->hdr));
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	709 ret = mail_transaction_log_file_read_hdr(file, TRUE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	710 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	711 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	712 if (ret <= 0) {
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	713 mail_transaction_log_file_free(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	714 return NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	715 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	716
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	717 mail_transaction_log_file_add_to_head(file);
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	718 return file;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	719 }
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	720
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	721 static struct mail_transaction_log_file *
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	722 mail_transaction_log_file_alloc_in_memory(struct mail_transaction_log *log)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	723 {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	724 struct mail_transaction_log_file *file;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	725
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	726 file = i_new(struct mail_transaction_log_file, 1);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	727 file->refcount = 1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	728 file->log = log;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	729 file->filepath = i_strdup("(in-memory transaction log file)");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	730 file->fd = -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	731
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	732 if (mail_transaction_log_init_hdr(log, &file->hdr) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	733 i_free(file);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	734 return NULL;
2449 1543762353aa Optimization for setting initial sync_offset from index. Timo Sirainen <tss@iki.fi> parents: 2440 diff changeset	735 }
1543762353aa Optimization for setting initial sync_offset from index. Timo Sirainen <tss@iki.fi> parents: 2440 diff changeset	736
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	737 file->buffer = buffer_create_dynamic(default_pool, 4096);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	738 file->buffer_offset = sizeof(file->hdr);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	739
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	740 mail_transaction_log_file_add_to_head(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	741 return file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	742 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	743
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	744 static struct mail_transaction_log_file *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	745 mail_transaction_log_file_open_or_create(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	746 const char *path)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	747 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	748 int fd;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	749
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	750 if (MAIL_INDEX_IS_IN_MEMORY(log->index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	751 return mail_transaction_log_file_alloc_in_memory(log);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	752
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	753 fd = open(path, O_RDWR);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	754 if (fd == -1) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	755 if (errno != ENOENT) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	756 mail_index_file_set_syscall_error(log->index, path,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	757 "open()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	758 return NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	759 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	760
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	761 fd = mail_transaction_log_file_create(log, path, 0, 0, 0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	762 if (fd == -1)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	763 return NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	764 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	765
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	766 return mail_transaction_log_file_fd_open_or_create(log, path, fd);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	767 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	768
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	769 static struct mail_transaction_log_file *
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	770 mail_transaction_log_file_open(struct mail_transaction_log *log,
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	771 const char *path)
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	772 {
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	773 struct mail_transaction_log_file *file;
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	774 int fd, ret;
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	775
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	776 fd = open(path, O_RDWR);
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	777 if (fd == -1) {
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	778 mail_index_file_set_syscall_error(log->index, path, "open()");
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	779 return NULL;
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	780 }
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	781
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	782 ret = mail_transaction_log_file_fd_open(log, &file, path, fd, TRUE);
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	783 if (ret <= 0) {
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	784 /* error / corrupted */
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	785 if (ret == 0)
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	786 mail_transaction_log_file_free(file);
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	787 return NULL;
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	788 }
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	789
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	790 mail_transaction_log_file_add_to_head(file);
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	791 return file;
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	792 }
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	793
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	794 void mail_transaction_logs_clean(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	795 {
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	796 struct mail_transaction_log_file *p, next;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	797
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	798 for (p = &log->tail; *p != NULL; ) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	799 if ((*p)->refcount != 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	800 p = &(*p)->next;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	801 else {
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	802 next = (*p)->next;
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	803 mail_transaction_log_file_free(*p);
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	804 *p = next;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	805 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	806 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	807 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	808
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	809 int mail_transaction_log_rotate(struct mail_transaction_log *log, bool lock)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	810 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	811 struct mail_transaction_log_file *file;
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	812 const char *path = log->head->filepath;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	813 struct stat st;
2008 23959140f4f5 some minor fixes Timo Sirainen <tss@iki.fi> parents: 2004 diff changeset	814 int fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	815
3173 d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	816 i_assert(log->head->locked);
d137899ea853 We could have gone past the transaction log view's boundaries if log was Timo Sirainen <tss@iki.fi> parents: 3170 diff changeset	817
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	818 if (MAIL_INDEX_IS_IN_MEMORY(log->index))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	819 file = mail_transaction_log_file_alloc_in_memory(log);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	820 else {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	821 if (fstat(log->head->fd, &st) < 0) {
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	822 mail_index_file_set_syscall_error(log->index, path,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	823 "fstat()");
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	824 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	825 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	826
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	827 fd = mail_transaction_log_file_create(log, path,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	828 st.st_dev, st.st_ino,
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	829 st.st_size);
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	830 if (fd == -1)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	831 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	832
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	833 file = mail_transaction_log_file_fd_open_or_create(log,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	834 path, fd);
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	835 if (file == NULL)
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	836 return -1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	837 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	838
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	839 if (lock) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	840 if (mail_transaction_log_file_lock(file) < 0) {
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	841 file->refcount--;
2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	842 mail_transaction_logs_clean(log);
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	843 return -1;
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	844 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	845 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	846 i_assert(file->locked == lock);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	847
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	848 if (--log->head->refcount == 0)
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	849 mail_transaction_logs_clean(log);
876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	850 else
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	851 mail_transaction_log_file_unlock(log->head);
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1937 diff changeset	852
2004 2ab29071a32c some fixes Timo Sirainen <tss@iki.fi> parents: 1976 diff changeset	853 i_assert(log->head != file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	854 log->head = file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	855 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	856 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	857
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	858 static int mail_transaction_log_refresh(struct mail_transaction_log *log,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	859 bool create_if_needed)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	860 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	861 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	862 struct stat st;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	863 const char *path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	864
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	865 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(log->head))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	866 return 0;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	867
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	868 path = t_strconcat(log->index->filepath,
3316 62d31be5ae60 s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. Timo Sirainen <tss@iki.fi> parents: 3243 diff changeset	869 MAIL_TRANSACTION_LOG_SUFFIX, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	870 if (stat(path, &st) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	871 mail_index_file_set_syscall_error(log->index, path, "stat()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	872 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	873 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	874
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	875 if (log->head != NULL &&
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	876 log->head->st_ino == st.st_ino &&
2318 4e925eb04b10 Use CMP_DEV_T() macro to compare struct stat.st_dev Timo Sirainen <tss@iki.fi> parents: 2285 diff changeset	877 CMP_DEV_T(log->head->st_dev, st.st_dev)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	878 /* same file */
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	879 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	880 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	881
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	882 file = create_if_needed ?
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	883 mail_transaction_log_file_open_or_create(log, path) :
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	884 mail_transaction_log_file_open(log, path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	885 if (file == NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	886 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	887
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	888 i_assert(!file->locked);
2095 92d8b70829c8 fixes Timo Sirainen <tss@iki.fi> parents: 2073 diff changeset	889
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	890 if (log->head != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	891 if (--log->head->refcount == 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	892 mail_transaction_logs_clean(log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	893 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	894
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	895 log->head = file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	896 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	897 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	898
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	899 int mail_transaction_log_file_find(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	900 uint32_t file_seq,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	901 struct mail_transaction_log_file **file_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	902 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	903 struct mail_transaction_log_file *file;
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	904 struct stat st;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	905 const char *path;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	906 int ret, fd;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	907
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	908 if (file_seq > log->head->hdr.file_seq) {
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	909 /* don't try to recreate log file if it gets lost. we're
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	910 already in trouble and with mmap_disable the creation
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	911 could cause a recursive mail_index_map() call */
afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	912 if (mail_transaction_log_refresh(log, FALSE) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	913 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	914 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	915
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	916 for (file = log->tail; file != NULL; file = file->next) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	917 if (file->hdr.file_seq == file_seq) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	918 *file_r = file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	919 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	920 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	921 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	922
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	923 /* see if we have it in log.2 file */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	924 path = t_strconcat(log->index->filepath,
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	925 MAIL_TRANSACTION_LOG_SUFFIX".2", NULL);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	926 fd = open(path, O_RDWR);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	927 if (fd == -1) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	928 if (errno == ENOENT)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	929 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	930
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	931 mail_index_file_set_syscall_error(log->index, path, "open()");
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	932 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	933 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	934
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	935 if (fstat(fd, &st) < 0) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	936 mail_index_file_set_syscall_error(log->index, path, "fstat()");
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	937 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	938 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	939
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	940 /* see if we have it already opened */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	941 for (file = log->tail; file != NULL; file = file->next) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	942 if (file->st_ino == st.st_ino &&
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	943 CMP_DEV_T(file->st_dev, st.st_dev)) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	944 if (close(fd) < 0)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	945 i_error("close() failed: %m");
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	946 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	947 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	948 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	949
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	950 ret = mail_transaction_log_file_fd_open(log, &file, path, fd, FALSE);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	951 if (ret <= 0) {
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	952 if (ret == 0) {
3368 1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	953 /* corrupted, delete it */
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	954 if (unlink(file->filepath) < 0 && errno != ENOENT) {
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	955 i_error("unlink(%s) failed: %m",
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	956 file->filepath);
1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	957 }
3833 318c70c1d4ec Beginnings of fallbacking to in-memory indexes when write fails with "out of Timo Sirainen <tss@iki.fi> parents: 3832 diff changeset	958 mail_transaction_log_file_free(file);
3320 0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	959 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	960 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	961 return -1;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	962 }
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	963
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	964 /* got it */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	965 mail_transaction_log_file_add_to_list(file);
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	966
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	967 /* but is it what we expected? */
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	968 if (file->hdr.file_seq != file_seq)
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	969 return 0;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	970
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	971 *file_r = file;
0403596c93fd index.log file is now rotated to .index.log.2 file. When log file is Timo Sirainen <tss@iki.fi> parents: 3317 diff changeset	972 return 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	973 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	974
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	975 static int
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	976 mail_transaction_log_file_sync(struct mail_transaction_log_file *file)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	977 {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	978 const struct mail_transaction_header *hdr;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	979 const void *data;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	980 size_t size;
3368 1069b25164d1 Delete transaction log.2 file if we detect it's corrupted. Timo Sirainen <tss@iki.fi> parents: 3350 diff changeset	981 uint32_t hdr_size = 0;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	982
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	983 data = buffer_get_data(file->buffer, &size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	984
3180 ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	985 if (file->sync_offset < file->buffer_offset)
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	986 file->sync_offset = file->buffer_offset;
ef16bb8091aa Several transaction log cleanups and fixes. Timo Sirainen <tss@iki.fi> parents: 3173 diff changeset	987
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	988 while (file->sync_offset - file->buffer_offset + sizeof(*hdr) <= size) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	989 hdr = CONST_PTR_OFFSET(data, file->sync_offset -
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	990 file->buffer_offset);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	991 hdr_size = mail_index_offset_to_uint32(hdr->size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	992 if (hdr_size == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	993 /* unfinished */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	994 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	995 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	996 if (hdr_size < sizeof(*hdr)) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	997 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	998 "hdr.size too small (%u)", hdr_size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	999 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1000 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1001
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1002 if (file->sync_offset - file->buffer_offset + hdr_size > size)
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1003 break;
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1004 file->sync_offset += hdr_size;
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1005 }
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1006
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1007 if (file->sync_offset - file->buffer_offset != size) {
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1008 /* record goes outside the file we've seen. or if
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1009 we're accessing the log file via unlocked mmaped
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1010 memory, it may be just that the memory was updated
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1011 after we checked the file size. */
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1012 if (file->locked \|\| file->mmap_base == NULL) {
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1013 mail_transaction_log_file_set_corrupted(file,
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1014 "hdr.size too large (%u)", hdr_size);
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1015 return -1;
3218 746c309e366c If transaction log record size is too large, mark the file corrupted. Timo Sirainen <tss@iki.fi> parents: 3180 diff changeset	1016 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1017 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1018 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1019 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1020
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1021 static int
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1022 mail_transaction_log_file_read(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1023 uoff_t offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1024 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1025 void *data;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1026 size_t size;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1027 uint32_t read_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1028 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1029
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1030 i_assert(file->mmap_base == NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1031
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1032 if (file->buffer != NULL && file->buffer_offset > offset) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1033 /* we have to insert missing data to beginning of buffer */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1034 size = file->buffer_offset - offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1035 buffer_copy(file->buffer, size, file->buffer, 0, (size_t)-1);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1036 file->buffer_offset -= size;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1037
1932 f04ff2a2ad5d fix Timo Sirainen <tss@iki.fi> parents: 1927 diff changeset	1038 data = buffer_get_space_unsafe(file->buffer, 0, size);
1933 bfdff8c75965 fixes for mmap_disable Timo Sirainen <tss@iki.fi> parents: 1932 diff changeset	1039 ret = pread_full(file->fd, data, size, offset);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1040 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1041 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1042 "Unexpected end of file");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1043 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1044 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1045 if (ret < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1046 if (errno == ESTALE) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1047 /* log file was deleted in NFS server,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1048 fail silently */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1049 return 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1050 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1051 mail_index_file_set_syscall_error(file->log->index,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1052 file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1053 "pread()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1054 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1055 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1056 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1057
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1058 if (file->buffer == NULL) {
2708 f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1059 file->buffer =
f1e9f3ec8135 Buffer API change: we no longer support limited sized buffers where Timo Sirainen <tss@iki.fi> parents: 2683 diff changeset	1060 buffer_create_dynamic(default_pool, LOG_PREFETCH);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1061 file->buffer_offset = offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1062 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1063
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1064 /* read all records */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1065 read_offset = file->buffer_offset + buffer_get_used_size(file->buffer);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1066
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1067 do {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1068 data = buffer_append_space_unsafe(file->buffer, LOG_PREFETCH);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1069 ret = pread(file->fd, data, LOG_PREFETCH, read_offset);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1070 if (ret > 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1071 read_offset += ret;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1072
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1073 size = read_offset - file->buffer_offset;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1074 buffer_set_used_size(file->buffer, size);
2746 bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1075 } while (ret > 0 \|\| (ret < 0 && errno == EINTR));
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1076
2746 bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1077 if (mail_transaction_log_file_sync(file) < 0)
bf2bdf2dd9ae Log syncing fixes Timo Sirainen <tss@iki.fi> parents: 2725 diff changeset	1078 return -1;
1919 4aeb7357951a locking fixes, mmap_disable=yes fixes Timo Sirainen <tss@iki.fi> parents: 1917 diff changeset	1079
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1080 if (ret == 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1081 /* EOF */
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1082 i_assert(file->sync_offset >= file->buffer_offset);
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1083 buffer_set_used_size(file->buffer,
43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1084 file->sync_offset - file->buffer_offset);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1085 return 1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1086 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1087
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1088 if (errno == ESTALE) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1089 /* log file was deleted in NFS server, fail silently */
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1090 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1091 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1092
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1093 mail_index_file_set_syscall_error(file->log->index, file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1094 "pread()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1095 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1096 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1097
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1098 int mail_transaction_log_file_map(struct mail_transaction_log_file *file,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1099 uoff_t start_offset, uoff_t end_offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1100 {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1101 struct mail_index *index = file->log->index;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1102 size_t size;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1103 struct stat st;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1104 int ret, use_mmap;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1105
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1106 i_assert(start_offset <= end_offset);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1107
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1108 if (file->hdr.indexid == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1109 /* corrupted */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1110 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1111 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1112
3243 40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	1113 if (MAIL_TRANSACTION_LOG_FILE_IN_MEMORY(file))
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	1114 return 1;
40b4ba3c55b8 In-memory indexes work again. Just pass dir as NULL to mail_index_alloc(). Timo Sirainen <tss@iki.fi> parents: 3218 diff changeset	1115
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	1116 if (start_offset < file->hdr.hdr_size) {
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1117 mail_transaction_log_file_set_corrupted(file,
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1118 "offset (%"PRIuUOFF_T") < header size (%u)",
3169 ec17099a6490 Added versioning to transaction log header. Added create_stamp to its Timo Sirainen <tss@iki.fi> parents: 3146 diff changeset	1119 start_offset, file->hdr.hdr_size);
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1120 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1121 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1122
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1123 /* with mmap_no_write we could alternatively just write to log with
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1124 msync() rather than pwrite(). but since there aren't many such OSes
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1125 left, it's easier to just use mmap_disable behavior with it */
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1126 use_mmap = !index->mmap_disable && !index->mmap_no_write;
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1127
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1128 if (file->buffer != NULL && file->buffer_offset <= start_offset) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1129 /* see if we already have it */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1130 size = buffer_get_used_size(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1131 if (file->buffer_offset + size >= end_offset)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1132 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1133 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1134
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1135 if (use_mmap) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1136 if (fstat(file->fd, &st) < 0) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1137 mail_index_file_set_syscall_error(index, file->filepath,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1138 "fstat()");
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1139 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1140 }
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1141 if (start_offset > (uoff_t)st.st_size) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1142 mail_transaction_log_file_set_corrupted(file,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1143 "start_offset (%"PRIuUOFF_T") > file size "
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1144 "(%"PRIuUOFF_T")", start_offset,
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1145 (uoff_t)st.st_size);
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1146 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1147 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1148
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1149 if (file->mmap_base != NULL &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1150 (uoff_t)st.st_size == file->mmap_size &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1151 file->buffer_offset <= start_offset &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1152 end_offset == (uoff_t)-1) {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1153 /* it's all mmaped already */
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1154 if (mail_transaction_log_file_sync(file) < 0)
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1155 return -1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1156 return 1;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1157 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1158 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1159
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1160 if (file->buffer != NULL &&
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1161 (file->mmap_base != NULL \|\| use_mmap)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1162 buffer_free(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1163 file->buffer = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1164 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1165 if (file->mmap_base != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1166 if (munmap(file->mmap_base, file->mmap_size) < 0) {
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1167 mail_index_file_set_syscall_error(index, file->filepath,
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1168 "munmap()");
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1169 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1170 file->mmap_base = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1171 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1172
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1173 if (!use_mmap) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1174 ret = mail_transaction_log_file_read(file, start_offset);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1175 if (ret <= 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1176 /* make sure we don't leave ourself in
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1177 inconsistent state */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1178 if (file->buffer != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1179 buffer_free(file->buffer);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1180 file->buffer = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1181 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1182 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1183 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1184 } else {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1185 file->mmap_size = st.st_size;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1186 file->mmap_base = mmap(NULL, file->mmap_size, PROT_READ,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1187 MAP_SHARED, file->fd, 0);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1188 if (file->mmap_base == MAP_FAILED) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1189 file->mmap_base = NULL;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1190 mail_index_file_set_syscall_error(index, file->filepath,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1191 "mmap()");
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1192 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1193 }
2440 0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1194
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1195 if (file->mmap_size > mmap_get_page_size()) {
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1196 if (madvise(file->mmap_base, file->mmap_size,
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1197 MADV_SEQUENTIAL) < 0) {
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1198 mail_index_file_set_syscall_error(index,
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1199 file->filepath, "madvise()");
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1200 }
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1201 }
0ac79623b58e Use madvise() for mmaped transaction log file. Timo Sirainen <tss@iki.fi> parents: 2410 diff changeset	1202
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1203 file->buffer = buffer_create_const_data(default_pool,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1204 file->mmap_base,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1205 file->mmap_size);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1206 file->buffer_offset = 0;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1207
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1208 if (mail_transaction_log_file_sync(file) < 0)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1209 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1210 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1211
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1212 if (end_offset != (uoff_t)-1 && end_offset > file->sync_offset) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1213 mail_transaction_log_file_set_corrupted(file,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1214 "end_offset (%"PRIuUOFF_T") > current sync_offset "
3342 43d4aad84752 Minor error checking fixes. Timo Sirainen <tss@iki.fi> parents: 3320 diff changeset	1215 "(%"PRIuUOFF_T")", end_offset, file->sync_offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1216 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1217 }
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1218
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1219 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1220 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1221
3146 9a8bf1c86699 Moved code into mail-transaction-log-append.c Timo Sirainen <tss@iki.fi> parents: 3138 diff changeset	1222 int mail_transaction_log_lock_head(struct mail_transaction_log *log)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1223 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1224 struct mail_transaction_log_file *file;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1225 int ret = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1226
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1227 /* we want to get the head file locked. this is a bit racy,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1228 since by the time we have it locked a new log file may have been
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1229 created.
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1230
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1231 creating new log file requires locking the head file, so if we
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1232 can lock it and don't see another file, we can be sure no-one is
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1233 creating a new log at the moment */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1234
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1235 for (;;) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1236 file = log->head;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1237 if (mail_transaction_log_file_lock(file) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1238 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1239
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1240 file->refcount++;
3555 afe8ab9072f0 When refreshing transaction log while trying to find new log file, don't Timo Sirainen <tss@iki.fi> parents: 3552 diff changeset	1241 ret = mail_transaction_log_refresh(log, TRUE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1242 if (--file->refcount == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1243 mail_transaction_logs_clean(log);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1244 file = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1245 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1246
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1247 if (ret == 0 && log->head == file) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1248 /* success */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1249 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1250 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1251
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1252 if (file != NULL)
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1253 mail_transaction_log_file_unlock(file);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1254
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1255 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1256 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1257
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1258 /* try again */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1259 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1260
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1261 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1262 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1263
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1264 int mail_transaction_log_sync_lock(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1265 uint32_t file_seq_r, uoff_t file_offset_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1266 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1267 i_assert(!log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1268
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1269 if (mail_transaction_log_lock_head(log) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1270 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1271
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1272 /* update sync_offset */
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1273 if (mail_transaction_log_file_map(log->head, log->head->sync_offset,
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1274 (uoff_t)-1) < 0) {
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1275 mail_transaction_log_file_unlock(log->head);
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1276 return -1;
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1277 }
fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1278
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1279 log->index->log_locked = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1280 *file_seq_r = log->head->hdr.file_seq;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1281 *file_offset_r = log->head->sync_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1282 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1283 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1284
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1285 void mail_transaction_log_sync_unlock(struct mail_transaction_log *log)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1286 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1287 i_assert(log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1288
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1289 log->index->log_locked = FALSE;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1290 mail_transaction_log_file_unlock(log->head);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1291 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1292
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1293 void mail_transaction_log_get_head(struct mail_transaction_log *log,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1294 uint32_t file_seq_r, uoff_t file_offset_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1295 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1296 i_assert(log->index->log_locked);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1297
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1298 *file_seq_r = log->head->hdr.file_seq;
2407 fed6d07bd8ee Transaction log file is now read-lockless. Timo Sirainen <tss@iki.fi> parents: 2318 diff changeset	1299 *file_offset_r = log->head->sync_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1300 }
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1301
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1302 bool mail_transaction_log_is_head_prev(struct mail_transaction_log *log,
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3833 diff changeset	1303 uint32_t file_seq, uoff_t file_offset)
3832 76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1304 {
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1305 return log->head->hdr.prev_file_seq == file_seq &&
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1306 log->head->hdr.prev_file_offset == file_offset;
76f4be7ae499 Handle missing/broken transaction logs better. Handle broken sync position Timo Sirainen <tss@iki.fi> parents: 3556 diff changeset	1307 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-index/mail-transaction-log.c @ 3879:928229f8b3e6 HEAD