Mercurial > dovecot > core-2.2

annotate src/lib-compression/istream-bzlib.c @ 22616:629f44740f50

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
cassandra: Include "prepared" when logging about prepared statement queries Mainly useful for debugging/testing.
author Timo Sirainen <timo.sirainen@dovecot.fi>
date Mon, 16 Oct 2017 15:12:12 +0300
parents bf1220873408
children cb108f786fb4
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
21390
2e2563132d5f Updated copyright notices to include the year 2017.
Stephan Bosch <stephan.bosch@dovecot.fi>
parents: 20829
diff changeset
1 /* Copyright (c) 2010-2017 Dovecot authors, see the included COPYING file */
7981
bb9d3aabcb61 zlib plugin supports now bzip2 also.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
2
bb9d3aabcb61 zlib plugin supports now bzip2 also.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
3 #include "lib.h"
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
4
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
5 #ifdef HAVE_BZLIB
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
6
13529
cf77e448295c Renamed lib/*-internal.h files to lib/*-private.h for consistency.
Timo Sirainen <tss@iki.fi>
parents: 13011
diff changeset
7 #include "istream-private.h"
7981
bb9d3aabcb61 zlib plugin supports now bzip2 also.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
8 #include "istream-zlib.h"
bb9d3aabcb61 zlib plugin supports now bzip2 also.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
9 #include <bzlib.h>
bb9d3aabcb61 zlib plugin supports now bzip2 also.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
10
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
11 #define CHUNK_SIZE (1024*64)
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
12
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
13 struct bzlib_istream {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
14 struct istream_private istream;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
15
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
16 bz_stream zs;
13986
c464072d05e0 zlib: Getting stream's size failed in some situations.
Timo Sirainen <tss@iki.fi>
parents: 13529
diff changeset
17 uoff_t eof_offset, stream_size;
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
18 size_t high_pos;
13011
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
19 struct stat last_parent_statbuf;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
20
10846
def12863b620 zlib: If reading corrupted compressed mail files, log an error.
Timo Sirainen <tss@iki.fi>
parents: 10810
diff changeset
21 unsigned int log_errors:1;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
22 unsigned int marked:1;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
23 unsigned int zs_closed:1;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
24 };
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
25
16020
6cabb95d32ec iostreams: Added close_parent flag to close() handler and clarified close/destroy APIs.
Timo Sirainen <tss@iki.fi>
parents: 15715
diff changeset
26 static void i_stream_bzlib_close(struct iostream_private *stream,
6cabb95d32ec iostreams: Added close_parent flag to close() handler and clarified close/destroy APIs.
Timo Sirainen <tss@iki.fi>
parents: 15715
diff changeset
27 bool close_parent)
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
28 {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
29 struct bzlib_istream *zstream = (struct bzlib_istream *)stream;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
30
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
31 if (!zstream->zs_closed) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
32 (void)BZ2_bzDecompressEnd(&zstream->zs);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
33 zstream->zs_closed = TRUE;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
34 }
16020
6cabb95d32ec iostreams: Added close_parent flag to close() handler and clarified close/destroy APIs.
Timo Sirainen <tss@iki.fi>
parents: 15715
diff changeset
35 if (close_parent)
6cabb95d32ec iostreams: Added close_parent flag to close() handler and clarified close/destroy APIs.
Timo Sirainen <tss@iki.fi>
parents: 15715
diff changeset
36 i_stream_close(zstream->istream.parent);
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
37 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
38
10850
257556760b62 zlib: Improved error message logging. Log input stream names.
Timo Sirainen <tss@iki.fi>
parents: 10846
diff changeset
39 static void bzlib_read_error(struct bzlib_istream *zstream, const char *error)
257556760b62 zlib: Improved error message logging. Log input stream names.
Timo Sirainen <tss@iki.fi>
parents: 10846
diff changeset
40 {
16773
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
41 io_stream_set_error(&zstream->istream.iostream,
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
42 "bzlib.read(%s): %s at %"PRIuUOFF_T,
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
43 i_stream_get_name(&zstream->istream.istream), error,
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
44 zstream->istream.abs_start_offset +
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
45 zstream->istream.istream.v_offset);
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
46 if (zstream->log_errors)
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
47 i_error("%s", zstream->istream.iostream.error);
10850
257556760b62 zlib: Improved error message logging. Log input stream names.
Timo Sirainen <tss@iki.fi>
parents: 10846
diff changeset
48 }
257556760b62 zlib: Improved error message logging. Log input stream names.
Timo Sirainen <tss@iki.fi>
parents: 10846
diff changeset
49
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
50 static ssize_t i_stream_bzlib_read(struct istream_private *stream)
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
51 {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
52 struct bzlib_istream *zstream = (struct bzlib_istream *)stream;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
53 const unsigned char *data;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
54 uoff_t high_offset;
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
55 size_t size, out_size;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
56 int ret;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
57
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
58 high_offset = stream->istream.v_offset + (stream->pos - stream->skip);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
59 if (zstream->eof_offset == high_offset) {
12080
0d5f21aeda68 zlib: Potential assert-crashfix.
Timo Sirainen <tss@iki.fi>
parents: 10864
diff changeset
60 i_assert(zstream->high_pos == 0 ||
0d5f21aeda68 zlib: Potential assert-crashfix.
Timo Sirainen <tss@iki.fi>
parents: 10864
diff changeset
61 zstream->high_pos == stream->pos);
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
62 stream->istream.eof = TRUE;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
63 return -1;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
64 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
65
10855
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
66 if (stream->pos < zstream->high_pos) {
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
67 /* we're here because we seeked back within the read buffer. */
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
68 ret = zstream->high_pos - stream->pos;
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
69 stream->pos = zstream->high_pos;
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
70 zstream->high_pos = 0;
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
71
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
72 if (zstream->eof_offset != (uoff_t)-1) {
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
73 high_offset = stream->istream.v_offset +
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
74 (stream->pos - stream->skip);
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
75 i_assert(zstream->eof_offset == high_offset);
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
76 stream->istream.eof = TRUE;
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
77 }
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
78 return ret;
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
79 }
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
80 zstream->high_pos = 0;
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
81
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
82 if (stream->pos + CHUNK_SIZE > stream->buffer_size) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
83 /* try to keep at least CHUNK_SIZE available */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
84 if (!zstream->marked && stream->skip > 0) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
85 /* don't try to keep anything cached if we don't
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
86 have a seek mark. */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
87 i_stream_compress(stream);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
88 }
20245
57078d325d01 global: Use i_stream_get_max_buffer_size() wherever possible
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20242
diff changeset
89 if (stream->buffer_size < i_stream_get_max_buffer_size(&stream->istream))
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
90 i_stream_grow_buffer(stream, CHUNK_SIZE);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
91
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
92 if (stream->pos == stream->buffer_size) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
93 if (stream->skip > 0) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
94 /* lose our buffer cache */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
95 i_stream_compress(stream);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
96 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
97
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
98 if (stream->pos == stream->buffer_size)
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
99 return -2; /* buffer full */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
100 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
101 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
102
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
103 if (i_stream_read_data(stream->parent, &data, &size, 0) < 0) {
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
104 if (stream->parent->stream_errno != 0) {
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
105 stream->istream.stream_errno =
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
106 stream->parent->stream_errno;
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
107 } else {
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
108 i_assert(stream->parent->eof);
16773
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
109 bzlib_read_error(zstream, "unexpected EOF");
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
110 stream->istream.stream_errno = EINVAL;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
111 }
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
112 return -1;
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
113 }
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
114 if (size == 0) {
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
115 /* no more input */
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
116 i_assert(!stream->istream.blocking);
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
117 return 0;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
118 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
119
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
120 zstream->zs.next_in = (char *)data;
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
121 zstream->zs.avail_in = size;
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
122
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
123 out_size = stream->buffer_size - stream->pos;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
124 zstream->zs.next_out = (char *)stream->w_buffer + stream->pos;
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
125 zstream->zs.avail_out = out_size;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
126 ret = BZ2_bzDecompress(&zstream->zs);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
127
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
128 out_size -= zstream->zs.avail_out;
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
129 stream->pos += out_size;
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
130
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
131 i_stream_skip(stream->parent, size - zstream->zs.avail_in);
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
132
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
133 switch (ret) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
134 case BZ_OK:
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
135 break;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
136 case BZ_PARAM_ERROR:
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
137 i_unreached();
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
138 case BZ_DATA_ERROR:
16773
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
139 bzlib_read_error(zstream, "corrupted data");
10846
def12863b620 zlib: If reading corrupted compressed mail files, log an error.
Timo Sirainen <tss@iki.fi>
parents: 10810
diff changeset
140 stream->istream.stream_errno = EINVAL;
def12863b620 zlib: If reading corrupted compressed mail files, log an error.
Timo Sirainen <tss@iki.fi>
parents: 10810
diff changeset
141 return -1;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
142 case BZ_DATA_ERROR_MAGIC:
16773
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
143 bzlib_read_error(zstream,
76d5e3c8cec3 iostreams: Set stream error string when it provides extra information.
Timo Sirainen <tss@iki.fi>
parents: 16315
diff changeset
144 "wrong magic in header (not bz2 file?)");
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
145 stream->istream.stream_errno = EINVAL;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
146 return -1;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
147 case BZ_MEM_ERROR:
10850
257556760b62 zlib: Improved error message logging. Log input stream names.
Timo Sirainen <tss@iki.fi>
parents: 10846
diff changeset
148 i_fatal_status(FATAL_OUTOFMEM, "bzlib.read(%s): Out of memory",
257556760b62 zlib: Improved error message logging. Log input stream names.
Timo Sirainen <tss@iki.fi>
parents: 10846
diff changeset
149 i_stream_get_name(&stream->istream));
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
150 case BZ_STREAM_END:
12224
e7768ec9d3de zlib: eof_offset wasn't always set right, causing assert-crashes.
Timo Sirainen <tss@iki.fi>
parents: 12223
diff changeset
151 zstream->eof_offset = stream->istream.v_offset +
e7768ec9d3de zlib: eof_offset wasn't always set right, causing assert-crashes.
Timo Sirainen <tss@iki.fi>
parents: 12223
diff changeset
152 (stream->pos - stream->skip);
13986
c464072d05e0 zlib: Getting stream's size failed in some situations.
Timo Sirainen <tss@iki.fi>
parents: 13529
diff changeset
153 zstream->stream_size = zstream->eof_offset;
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
154 if (out_size == 0) {
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
155 stream->istream.eof = TRUE;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
156 return -1;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
157 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
158 break;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
159 default:
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
160 i_fatal("BZ2_bzDecompress() failed with %d", ret);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
161 }
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
162 if (out_size == 0) {
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
163 /* read more input */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
164 return i_stream_bzlib_read(stream);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
165 }
16315
ba63c27667ff istream-[b]zlib: Don't break if parent stream gets seeked in the middle of reads.
Timo Sirainen <tss@iki.fi>
parents: 16020
diff changeset
166 return out_size;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
167 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
168
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
169 static void i_stream_bzlib_init(struct bzlib_istream *zstream)
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
170 {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
171 int ret;
7981
bb9d3aabcb61 zlib plugin supports now bzip2 also.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
172
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
173 ret = BZ2_bzDecompressInit(&zstream->zs, 0, 0);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
174 switch (ret) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
175 case BZ_OK:
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
176 break;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
177 case BZ_MEM_ERROR:
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
178 i_fatal_status(FATAL_OUTOFMEM, "bzlib: Out of memory");
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
179 case BZ_CONFIG_ERROR:
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
180 i_fatal("Wrong bzlib library version (broken compilation)");
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
181 case BZ_PARAM_ERROR:
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
182 i_fatal("bzlib: Invalid parameters");
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
183 default:
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
184 i_fatal("BZ2_bzDecompressInit() failed with %d", ret);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
185 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
186 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
187
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
188 static void i_stream_bzlib_reset(struct bzlib_istream *zstream)
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
189 {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
190 struct istream_private *stream = &zstream->istream;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
191
10855
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
192 i_stream_seek(stream->parent, stream->parent_start_offset);
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
193 zstream->eof_offset = (uoff_t)-1;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
194 zstream->zs.next_in = NULL;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
195 zstream->zs.avail_in = 0;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
196
10864
d05561b0c257 zlib: Fixed seeking in zlib/bzlib input stream.
Timo Sirainen <tss@iki.fi>
parents: 10855
diff changeset
197 stream->parent_expected_offset = stream->parent_start_offset;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
198 stream->skip = stream->pos = 0;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
199 stream->istream.v_offset = 0;
10855
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
200 zstream->high_pos = 0;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
201
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
202 (void)BZ2_bzDecompressEnd(&zstream->zs);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
203 i_stream_bzlib_init(zstream);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
204 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
205
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
206 static void
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
207 i_stream_bzlib_seek(struct istream_private *stream, uoff_t v_offset, bool mark)
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
208 {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
209 struct bzlib_istream *zstream = (struct bzlib_istream *) stream;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
210 uoff_t start_offset = stream->istream.v_offset - stream->skip;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
211
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
212 if (v_offset < start_offset) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
213 /* have to seek backwards */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
214 i_stream_bzlib_reset(zstream);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
215 start_offset = 0;
10855
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
216 } else if (zstream->high_pos != 0) {
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
217 stream->pos = zstream->high_pos;
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
218 zstream->high_pos = 0;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
219 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
220
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
221 if (v_offset <= start_offset + stream->pos) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
222 /* seeking backwards within what's already cached */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
223 stream->skip = v_offset - start_offset;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
224 stream->istream.v_offset = v_offset;
10855
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
225 zstream->high_pos = stream->pos;
681e1c702899 zlib: Several fixes to zlib/bzlib istreams.
Timo Sirainen <tss@iki.fi>
parents: 10850
diff changeset
226 stream->pos = stream->skip;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
227 } else {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
228 /* read and cache forward */
22500
bf1220873408 lib-compression: Fix assert in i_stream_zlib_seek
Anton Yuzhaninov <citrin@citrin.ru>
parents: 21390
diff changeset
229 ssize_t ret;
20828
d7bed2a85e99 global: Make sure i_stream_read() calls handle 0 and -2 return values correctly.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20245
diff changeset
230
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
231 do {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
232 size_t avail = stream->pos - stream->skip;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
233
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
234 if (stream->istream.v_offset + avail >= v_offset) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
235 i_stream_skip(&stream->istream,
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
236 v_offset -
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
237 stream->istream.v_offset);
22500
bf1220873408 lib-compression: Fix assert in i_stream_zlib_seek
Anton Yuzhaninov <citrin@citrin.ru>
parents: 21390
diff changeset
238 ret = -1;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
239 break;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
240 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
241
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
242 i_stream_skip(&stream->istream, avail);
20828
d7bed2a85e99 global: Make sure i_stream_read() calls handle 0 and -2 return values correctly.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20245
diff changeset
243 } while ((ret = i_stream_read(&stream->istream)) > 0);
d7bed2a85e99 global: Make sure i_stream_read() calls handle 0 and -2 return values correctly.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20245
diff changeset
244 i_assert(ret == -1);
7981
bb9d3aabcb61 zlib plugin supports now bzip2 also.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
245
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
246 if (stream->istream.v_offset != v_offset) {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
247 /* some failure, we've broken it */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
248 if (stream->istream.stream_errno != 0) {
10850
257556760b62 zlib: Improved error message logging. Log input stream names.
Timo Sirainen <tss@iki.fi>
parents: 10846
diff changeset
249 i_error("bzlib_istream.seek(%s) failed: %s",
257556760b62 zlib: Improved error message logging. Log input stream names.
Timo Sirainen <tss@iki.fi>
parents: 10846
diff changeset
250 i_stream_get_name(&stream->istream),
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
251 strerror(stream->istream.stream_errno));
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
252 i_stream_close(&stream->istream);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
253 } else {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
254 /* unexpected EOF. allow it since we may just
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
255 want to check if there's anything.. */
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
256 i_assert(stream->istream.eof);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
257 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
258 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
259 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
260
12223
138d848b1e22 zlib: Fixed istream-bzlib also with mark=TRUE seeking
Timo Sirainen <tss@iki.fi>
parents: 12080
diff changeset
261 if (mark)
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
262 zstream->marked = TRUE;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
263 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
264
14964
6d2be8d8891c i_stream_stat() API changed.
Timo Sirainen <tss@iki.fi>
parents: 14739
diff changeset
265 static int
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
266 i_stream_bzlib_stat(struct istream_private *stream, bool exact)
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
267 {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
268 struct bzlib_istream *zstream = (struct bzlib_istream *) stream;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
269 const struct stat *st;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
270 size_t size;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
271
18896
ac64c307d179 istream filters: If parent's i_stream_stat() fails, copy the stream_errno.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
272 if (i_stream_stat(stream->parent, exact, &st) < 0) {
ac64c307d179 istream filters: If parent's i_stream_stat() fails, copy the stream_errno.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
273 stream->istream.stream_errno = stream->parent->stream_errno;
14964
6d2be8d8891c i_stream_stat() API changed.
Timo Sirainen <tss@iki.fi>
parents: 14739
diff changeset
274 return -1;
18896
ac64c307d179 istream filters: If parent's i_stream_stat() fails, copy the stream_errno.
Timo Sirainen <tss@iki.fi>
parents: 18137
diff changeset
275 }
14964
6d2be8d8891c i_stream_stat() API changed.
Timo Sirainen <tss@iki.fi>
parents: 14739
diff changeset
276 stream->statbuf = *st;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
277
13010
15a0687ec9d0 istream-[b]zlib: stat(exact=FALSE) should always return the same value if file hasn't changed.
Timo Sirainen <tss@iki.fi>
parents: 12782
diff changeset
278 /* when exact=FALSE always return the parent stat's size, even if we
15a0687ec9d0 istream-[b]zlib: stat(exact=FALSE) should always return the same value if file hasn't changed.
Timo Sirainen <tss@iki.fi>
parents: 12782
diff changeset
279 know the exact value. this is necessary because otherwise e.g. mbox
15a0687ec9d0 istream-[b]zlib: stat(exact=FALSE) should always return the same value if file hasn't changed.
Timo Sirainen <tss@iki.fi>
parents: 12782
diff changeset
280 code can see two different values and think that a compressed mbox
15a0687ec9d0 istream-[b]zlib: stat(exact=FALSE) should always return the same value if file hasn't changed.
Timo Sirainen <tss@iki.fi>
parents: 12782
diff changeset
281 file keeps changing. */
15a0687ec9d0 istream-[b]zlib: stat(exact=FALSE) should always return the same value if file hasn't changed.
Timo Sirainen <tss@iki.fi>
parents: 12782
diff changeset
282 if (!exact)
14964
6d2be8d8891c i_stream_stat() API changed.
Timo Sirainen <tss@iki.fi>
parents: 14739
diff changeset
283 return 0;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
284
13986
c464072d05e0 zlib: Getting stream's size failed in some situations.
Timo Sirainen <tss@iki.fi>
parents: 13529
diff changeset
285 if (zstream->stream_size == (uoff_t)-1) {
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
286 uoff_t old_offset = stream->istream.v_offset;
20828
d7bed2a85e99 global: Make sure i_stream_read() calls handle 0 and -2 return values correctly.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20245
diff changeset
287 ssize_t ret;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
288
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
289 do {
14683
9646f80ac3e9 Added i_stream_get_data_size(). Used it where possible.
Timo Sirainen <tss@iki.fi>
parents: 14133
diff changeset
290 size = i_stream_get_data_size(&stream->istream);
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
291 i_stream_skip(&stream->istream, size);
20828
d7bed2a85e99 global: Make sure i_stream_read() calls handle 0 and -2 return values correctly.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20245
diff changeset
292 } while ((ret = i_stream_read(&stream->istream)) > 0);
d7bed2a85e99 global: Make sure i_stream_read() calls handle 0 and -2 return values correctly.
Timo Sirainen <timo.sirainen@dovecot.fi>
parents: 20245
diff changeset
293 i_assert(ret == -1);
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
294
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
295 i_stream_seek(&stream->istream, old_offset);
13986
c464072d05e0 zlib: Getting stream's size failed in some situations.
Timo Sirainen <tss@iki.fi>
parents: 13529
diff changeset
296 if (zstream->stream_size == (uoff_t)-1)
14964
6d2be8d8891c i_stream_stat() API changed.
Timo Sirainen <tss@iki.fi>
parents: 14739
diff changeset
297 return -1;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
298 }
13986
c464072d05e0 zlib: Getting stream's size failed in some situations.
Timo Sirainen <tss@iki.fi>
parents: 13529
diff changeset
299 stream->statbuf.st_size = zstream->stream_size;
14964
6d2be8d8891c i_stream_stat() API changed.
Timo Sirainen <tss@iki.fi>
parents: 14739
diff changeset
300 return 0;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
301 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
302
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
303 static void i_stream_bzlib_sync(struct istream_private *stream)
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
304 {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
305 struct bzlib_istream *zstream = (struct bzlib_istream *) stream;
13011
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
306 const struct stat *st;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
307
14964
6d2be8d8891c i_stream_stat() API changed.
Timo Sirainen <tss@iki.fi>
parents: 14739
diff changeset
308 if (i_stream_stat(stream->parent, FALSE, &st) < 0) {
13011
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
309 if (memcmp(&zstream->last_parent_statbuf,
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
310 st, sizeof(*st)) == 0) {
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
311 /* a compressed file doesn't change unexpectedly,
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
312 don't clear our caches unnecessarily */
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
313 return;
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
314 }
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
315 zstream->last_parent_statbuf = *st;
66ec075a49d3 zlib: Don't unnecessarily reset istream caches.
Timo Sirainen <tss@iki.fi>
parents: 13010
diff changeset
316 }
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
317 i_stream_bzlib_reset(zstream);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
318 }
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
319
10846
def12863b620 zlib: If reading corrupted compressed mail files, log an error.
Timo Sirainen <tss@iki.fi>
parents: 10810
diff changeset
320 struct istream *i_stream_create_bz2(struct istream *input, bool log_errors)
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
321 {
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
322 struct bzlib_istream *zstream;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
323
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
324 zstream = i_new(struct bzlib_istream, 1);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
325 zstream->eof_offset = (uoff_t)-1;
13986
c464072d05e0 zlib: Getting stream's size failed in some situations.
Timo Sirainen <tss@iki.fi>
parents: 13529
diff changeset
326 zstream->stream_size = (uoff_t)-1;
10846
def12863b620 zlib: If reading corrupted compressed mail files, log an error.
Timo Sirainen <tss@iki.fi>
parents: 10810
diff changeset
327 zstream->log_errors = log_errors;
10693
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
328
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
329 i_stream_bzlib_init(zstream);
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
330
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
331 zstream->istream.iostream.close = i_stream_bzlib_close;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
332 zstream->istream.max_buffer_size = input->real_stream->max_buffer_size;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
333 zstream->istream.read = i_stream_bzlib_read;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
334 zstream->istream.seek = i_stream_bzlib_seek;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
335 zstream->istream.stat = i_stream_bzlib_stat;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
336 zstream->istream.sync = i_stream_bzlib_sync;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
337
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
338 zstream->istream.istream.readable_fd = FALSE;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
339 zstream->istream.istream.blocking = input->blocking;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
340 zstream->istream.istream.seekable = input->seekable;
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
341
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
342 return i_stream_create(&zstream->istream, input,
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
343 i_stream_get_fd(input));
74d9dbee1399 zlib: Reimplemented gz/bz2 input streams by using the uncompression functions directly.
Timo Sirainen <tss@iki.fi>
parents: 10582
diff changeset
344 }
7981
bb9d3aabcb61 zlib plugin supports now bzip2 also.
Timo Sirainen <tss@iki.fi>
parents:
diff changeset
345 #endif