Mercurial > hg > pyhgsh

annotate mercurial/revlog.py @ 1403:bc3e66edb04c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
lazyindex fix, make load handle negative indexes properly.
author Eric Hopper <hopper@omnifarious.org>
date Tue, 18 Oct 2005 20:05:54 -0700
parents 9d2c2e6b32b5
children 0847c45ffee6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
1 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2 revlog.py - storage back-end for mercurial
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
3
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
4 This provides efficient delta storage with O(1) retrieve and append
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
5 and O(changes) merge between branches
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
6
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
7 Copyright 2005 Matt Mackall <mpm@selenic.com>
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
8
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
9 This software may be used and distributed according to the terms
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
10 of the GNU General Public License, incorporated herein by reference.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
11 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
12
1089
142b5d5ec9cc Break apart hg.py
mpm@selenic.com
parents: 1083
diff changeset
13 from node import *
1400
cf9a1233738a i18n first part: make '_' available for files who need it
Benoit Boissinot <benoit.boissinot@ens-lyon.org
parents: 1393
diff changeset
14 from i18n import gettext as _
1322
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
15 from demandload import demandload
1325
57220daf40e9 Move urllib error handling from revlog into statichttprepo, where it belongs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1322
diff changeset
16 demandload(globals(), "binascii errno heapq mdiff sha struct zlib")
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
17
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
18 def hash(text, p1, p2):
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
19 """generate a hash from the given text and its parent hashes
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
20
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
21 This hash combines both the current file contents and its history
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
22 in a manner that makes it easy to distinguish nodes with the same
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
23 content in the revision graph.
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
24 """
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
25 l = [p1, p2]
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
26 l.sort()
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
27 s = sha.new(l[0])
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
28 s.update(l[1])
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
29 s.update(text)
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
30 return s.digest()
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
31
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
32 def compress(text):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
33 """ generate a possibly-compressed representation of text """
112
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
34 if not text: return text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
35 if len(text) < 44:
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
36 if text[0] == '\0': return text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
37 return 'u' + text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
38 bin = zlib.compress(text)
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
39 if len(bin) > len(text):
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
40 if text[0] == '\0': return text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
41 return 'u' + text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
42 return bin
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
43
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
44 def decompress(bin):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
45 """ decompress the given input """
112
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
46 if not bin: return bin
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
47 t = bin[0]
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
48 if t == '\0': return bin
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
49 if t == 'x': return zlib.decompress(bin)
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
50 if t == 'u': return bin[1:]
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
51 raise RevlogError(_("unknown compression type %s") % t)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
52
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
53 indexformat = ">4l20s20s20s"
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
54
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
55 class lazyparser:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
56 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
57 this class avoids the need to parse the entirety of large indices
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
58
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
59 By default we parse and load 1000 entries at a time.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
60
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
61 If no position is specified, we load the whole index, and replace
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
62 the lazy objects in revlog with the underlying objects for
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
63 efficiency in cases where we look at most of the nodes.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
64 """
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
65 def __init__(self, data, revlog):
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
66 self.data = data
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
67 self.s = struct.calcsize(indexformat)
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
68 self.l = len(data)/self.s
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
69 self.index = [None] * self.l
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
70 self.map = {nullid: -1}
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
71 self.all = 0
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
72 self.revlog = revlog
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
73
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
74 def load(self, pos=None):
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
75 if self.all: return
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
76 if pos is not None:
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
77 block = pos / 1000
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
78 i = block * 1000
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
79 end = min(self.l, i + 1000)
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
80 else:
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
81 self.all = 1
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
82 i = 0
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
83 end = self.l
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
84 self.revlog.index = self.index
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
85 self.revlog.nodemap = self.map
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
86
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
87 while i < end:
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
88 d = self.data[i * self.s: (i + 1) * self.s]
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
89 e = struct.unpack(indexformat, d)
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
90 self.index[i] = e
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
91 self.map[e[6]] = i
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
92 i += 1
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
93
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
94 class lazyindex:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
95 """a lazy version of the index array"""
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
96 def __init__(self, parser):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
97 self.p = parser
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
98 def __len__(self):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
99 return len(self.p.index)
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
100 def load(self, pos):
1403
bc3e66edb04c lazyindex fix, make load handle negative indexes properly.
Eric Hopper <hopper@omnifarious.org>
parents: 1402
diff changeset
101 if pos < 0:
bc3e66edb04c lazyindex fix, make load handle negative indexes properly.
Eric Hopper <hopper@omnifarious.org>
parents: 1402
diff changeset
102 pos += len(self.p.index)
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
103 self.p.load(pos)
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
104 return self.p.index[pos]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
105 def __getitem__(self, pos):
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
106 return self.p.index[pos] or self.load(pos)
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
107 def append(self, e):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
108 self.p.index.append(e)
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
109
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
110 class lazymap:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
111 """a lazy version of the node map"""
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
112 def __init__(self, parser):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
113 self.p = parser
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
114 def load(self, key):
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
115 if self.p.all: return
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
116 n = self.p.data.find(key)
1214
34706a835d4a Smarter handling of revlog key errors
mpm@selenic.com
parents: 1201
diff changeset
117 if n < 0:
34706a835d4a Smarter handling of revlog key errors
mpm@selenic.com
parents: 1201
diff changeset
118 raise KeyError(key)
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
119 pos = n / self.p.s
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
120 self.p.load(pos)
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
121 def __contains__(self, key):
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
122 self.p.load()
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
123 return key in self.p.map
97
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
124 def __iter__(self):
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
125 yield nullid
97
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
126 for i in xrange(self.p.l):
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
127 try:
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
128 yield self.p.index[i][6]
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
129 except:
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
130 self.p.load(i)
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
131 yield self.p.index[i][6]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
132 def __getitem__(self, key):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
133 try:
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
134 return self.p.map[key]
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
135 except KeyError:
86
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
136 try:
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
137 self.load(key)
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
138 return self.p.map[key]
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
139 except KeyError:
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
140 raise KeyError("node " + hex(key))
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
141 def __setitem__(self, key, val):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
142 self.p.map[key] = val
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
143
1073
7b35a980b982 [PATCH] raise exceptions with Exception subclasses
Bart Trojanowski <bart@jukie.net>
parents: 1062
diff changeset
144 class RevlogError(Exception): pass
7b35a980b982 [PATCH] raise exceptions with Exception subclasses
Bart Trojanowski <bart@jukie.net>
parents: 1062
diff changeset
145
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
146 class revlog:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
147 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
148 the underlying revision storage object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
149
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
150 A revlog consists of two parts, an index and the revision data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
151
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
152 The index is a file with a fixed record size containing
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
153 information on each revision, includings its nodeid (hash), the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
154 nodeids of its parents, the position and offset of its data within
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
155 the data file, and the revision it's based on. Finally, each entry
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
156 contains a linkrev entry that can serve as a pointer to external
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
157 data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
158
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
159 The revision data itself is a linear collection of data chunks.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
160 Each chunk represents a revision and is usually represented as a
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
161 delta against the previous chunk. To bound lookup time, runs of
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
162 deltas are limited to about 2 times the length of the original
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
163 version data. This makes retrieval of a version proportional to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
164 its size, or O(1) relative to the number of revisions.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
165
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
166 Both pieces of the revlog are written to in an append-only
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
167 fashion, which means we never need to rewrite a file to insert or
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
168 remove data, and can use some simple techniques to avoid the need
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
169 for locking while reading.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
170 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
171 def __init__(self, opener, indexfile, datafile):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
172 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
173 create a revlog object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
174
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
175 opener is a function that abstracts the file opening operation
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
176 and can be used to implement COW semantics or the like.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
177 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
178 self.indexfile = indexfile
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
179 self.datafile = datafile
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
180 self.opener = opener
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
181 self.cache = None
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
182
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
183 try:
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
184 i = self.opener(self.indexfile).read()
1322
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
185 except IOError, inst:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
186 if inst.errno != errno.ENOENT:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
187 raise
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
188 i = ""
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
189
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
190 if len(i) > 10000:
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
191 # big index, let's parse it on demand
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
192 parser = lazyparser(i, self)
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
193 self.index = lazyindex(parser)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
194 self.nodemap = lazymap(parser)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
195 else:
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
196 s = struct.calcsize(indexformat)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
197 l = len(i) / s
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
198 self.index = [None] * l
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
199 m = [None] * l
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
200
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
201 n = 0
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
202 for f in xrange(0, len(i), s):
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
203 # offset, size, base, linkrev, p1, p2, nodeid
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
204 e = struct.unpack(indexformat, i[f:f + s])
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
205 m[n] = (e[6], n)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
206 self.index[n] = e
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
207 n += 1
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
208
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
209 self.nodemap = dict(m)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
210 self.nodemap[nullid] = -1
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
211
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
212 def tip(self): return self.node(len(self.index) - 1)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
213 def count(self): return len(self.index)
26
9cf83bf9ad38 Simplify integrity checking
mpm@selenic.com
parents: 14
diff changeset
214 def node(self, rev): return (rev < 0) and nullid or self.index[rev][6]
1201
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
215 def rev(self, node):
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
216 try:
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
217 return self.nodemap[node]
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
218 except KeyError:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
219 raise RevlogError(_('%s: no node %s') % (self.indexfile, hex(node)))
1201
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
220 def linkrev(self, node): return self.index[self.rev(node)][3]
2
ecf3fd948051 Handle nullid better for ancestor
mpm@selenic.com
parents: 0
diff changeset
221 def parents(self, node):
ecf3fd948051 Handle nullid better for ancestor
mpm@selenic.com
parents: 0
diff changeset
222 if node == nullid: return (nullid, nullid)
1201
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
223 return self.index[self.rev(node)][4:6]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
224
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
225 def start(self, rev): return self.index[rev][0]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
226 def length(self, rev): return self.index[rev][1]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
227 def end(self, rev): return self.start(rev) + self.length(rev)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
228 def base(self, rev): return self.index[rev][2]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
229
1074
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
230 def reachable(self, rev, stop=None):
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
231 reachable = {}
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
232 visit = [rev]
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
233 reachable[rev] = 1
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
234 if stop:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
235 stopn = self.rev(stop)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
236 else:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
237 stopn = 0
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
238 while visit:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
239 n = visit.pop(0)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
240 if n == stop:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
241 continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
242 if n == nullid:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
243 continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
244 for p in self.parents(n):
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
245 if self.rev(p) < stopn:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
246 continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
247 if p not in reachable:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
248 reachable[p] = 1
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
249 visit.append(p)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
250 return reachable
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
251
902
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
252 def heads(self, stop=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
253 """return the list of all nodes that have no children"""
221
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
254 p = {}
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
255 h = []
902
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
256 stoprev = 0
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
257 if stop and stop in self.nodemap:
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
258 stoprev = self.rev(stop)
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
259
243
9a9ea2d1d3c4 fix heads for rev 0
mpm@selenic.com
parents: 241
diff changeset
260 for r in range(self.count() - 1, -1, -1):
221
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
261 n = self.node(r)
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
262 if n not in p:
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
263 h.append(n)
902
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
264 if n == stop:
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
265 break
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
266 if r < stoprev:
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
267 break
221
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
268 for pn in self.parents(n):
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
269 p[pn] = 1
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
270 return h
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
271
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
272 def children(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
273 """find the children of a given node"""
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
274 c = []
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
275 p = self.rev(node)
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
276 for r in range(p + 1, self.count()):
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
277 n = self.node(r)
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
278 for pn in self.parents(n):
854
473c030d34a6 Fixed revlog.children.
Tristan Wibberley <tristan@wibberley.org>
parents: 655
diff changeset
279 if pn == node:
473c030d34a6 Fixed revlog.children.
Tristan Wibberley <tristan@wibberley.org>
parents: 655
diff changeset
280 c.append(n)
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
281 continue
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
282 elif pn == nullid:
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
283 continue
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
284 return c
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
285
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
286 def lookup(self, id):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
287 """locate a node based on revision number or subset of hex nodeid"""
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
288 try:
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
289 rev = int(id)
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
290 if str(rev) != id: raise ValueError
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
291 if rev < 0: rev = self.count() + rev
476
0a338d506268 Really _call_ method revlog.count in revlog.lookup()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 469
diff changeset
292 if rev < 0 or rev >= self.count(): raise ValueError
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
293 return self.node(rev)
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
294 except (ValueError, OverflowError):
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
295 c = []
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
296 for n in self.nodemap:
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
297 if hex(n).startswith(id):
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
298 c.append(n)
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
299 if len(c) > 1: raise RevlogError(_("Ambiguous identifier"))
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
300 if len(c) < 1: raise RevlogError(_("No match found"))
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
301 return c[0]
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
302
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
303 return None
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
304
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
305 def diff(self, a, b):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
306 """return a delta between two revisions"""
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
307 return mdiff.textdiff(a, b)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
308
73
ee1cbe841e01 Change revlog to use new patch code
mpm@selenic.com
parents: 71
diff changeset
309 def patches(self, t, pl):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
310 """apply a list of patches to a string"""
73
ee1cbe841e01 Change revlog to use new patch code
mpm@selenic.com
parents: 71
diff changeset
311 return mdiff.patches(t, pl)
ee1cbe841e01 Change revlog to use new patch code
mpm@selenic.com
parents: 71
diff changeset
312
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
313 def delta(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
314 """return or calculate a delta between a node and its predecessor"""
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
315 r = self.rev(node)
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
316 b = self.base(r)
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
317 if r == b:
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
318 return self.diff(self.revision(self.node(r - 1)),
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
319 self.revision(node))
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
320 else:
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
321 f = self.opener(self.datafile)
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
322 f.seek(self.start(r))
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
323 data = f.read(self.length(r))
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
324 return decompress(data)
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
325
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
326 def revision(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
327 """return an uncompressed revision of a given"""
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
328 if node == nullid: return ""
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
329 if self.cache and self.cache[0] == node: return self.cache[2]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
330
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
331 # look up what we need to read
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
332 text = None
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
333 rev = self.rev(node)
117
2ac722ad1a9d Make revision code slightly faster
mpm@selenic.com
parents: 116
diff changeset
334 start, length, base, link, p1, p2, node = self.index[rev]
2ac722ad1a9d Make revision code slightly faster
mpm@selenic.com
parents: 116
diff changeset
335 end = start + length
2ac722ad1a9d Make revision code slightly faster
mpm@selenic.com
parents: 116
diff changeset
336 if base != rev: start = self.start(base)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
337
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
338 # do we have useful data cached?
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
339 if self.cache and self.cache[1] >= base and self.cache[1] < rev:
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
340 base = self.cache[1]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
341 start = self.start(base + 1)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
342 text = self.cache[2]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
343 last = 0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
344
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
345 f = self.opener(self.datafile)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
346 f.seek(start)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
347 data = f.read(end - start)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
348
651
08b65d66f3e7 Fix an odd revlog bug
Matt Mackall <mpm@selenic.com>
parents: 644
diff changeset
349 if text is None:
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
350 last = self.length(base)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
351 text = decompress(data[:last])
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
352
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
353 bins = []
64
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
354 for r in xrange(base + 1, rev + 1):
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
355 s = self.length(r)
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
356 bins.append(decompress(data[last:last + s]))
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
357 last = last + s
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
358
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
359 text = mdiff.patches(text, bins)
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
360
26
9cf83bf9ad38 Simplify integrity checking
mpm@selenic.com
parents: 14
diff changeset
361 if node != hash(text, p1, p2):
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
362 raise RevlogError(_("integrity check failed on %s:%d")
98
3dde7c87e36d Add paranoia to diff code
mpm@selenic.com
parents: 97
diff changeset
363 % (self.datafile, rev))
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
364
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
365 self.cache = (node, rev, text)
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
366 return text
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
367
644
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
368 def addrevision(self, text, transaction, link, p1=None, p2=None, d=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
369 """add a revision to the log
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
370
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
371 text - the revision data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
372 transaction - the transaction object used for rollback
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
373 link - the linkrev data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
374 p1, p2 - the parent nodeids of the revision
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
375 d - an optional precomputed delta
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
376 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
377 if text is None: text = ""
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
378 if p1 is None: p1 = self.tip()
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
379 if p2 is None: p2 = nullid
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
380
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
381 node = hash(text, p1, p2)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
382
301
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
383 if node in self.nodemap:
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
384 return node
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
385
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
386 n = self.count()
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
387 t = n - 1
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
388
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
389 if n:
64
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
390 base = self.base(t)
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
391 start = self.start(base)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
392 end = self.end(t)
644
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
393 if not d:
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
394 prev = self.revision(self.tip())
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
395 d = self.diff(prev, text)
98
3dde7c87e36d Add paranoia to diff code
mpm@selenic.com
parents: 97
diff changeset
396 data = compress(d)
64
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
397 dist = end - start + len(data)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
398
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
399 # full versions are inserted when the needed deltas
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
400 # become comparable to the uncompressed text
64
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
401 if not n or dist > len(text) * 2:
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
402 data = compress(text)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
403 base = n
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
404 else:
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
405 base = self.base(t)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
406
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
407 offset = 0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
408 if t >= 0:
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
409 offset = self.end(t)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
410
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
411 e = (offset, len(data), base, link, p1, p2, node)
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
412
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
413 self.index.append(e)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
414 self.nodemap[node] = n
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
415 entry = struct.pack(indexformat, *e)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
416
26
9cf83bf9ad38 Simplify integrity checking
mpm@selenic.com
parents: 14
diff changeset
417 transaction.add(self.datafile, e[0])
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
418 self.opener(self.datafile, "a").write(data)
41
df3f46253878 Fix truncate logic for indices again
mpm@selenic.com
parents: 36
diff changeset
419 transaction.add(self.indexfile, n * len(entry))
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
420 self.opener(self.indexfile, "a").write(entry)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
421
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
422 self.cache = (node, n, text)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
423 return node
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
424
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
425 def ancestor(self, a, b):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
426 """calculate the least common ancestor of nodes a and b"""
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
427 # calculate the distance of every node from root
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
428 dist = {nullid: 0}
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
429 for i in xrange(self.count()):
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
430 n = self.node(i)
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
431 p1, p2 = self.parents(n)
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
432 dist[n] = max(dist[p1], dist[p2]) + 1
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
433
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
434 # traverse ancestors in order of decreasing distance from root
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
435 def ancestors(node):
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
436 # we store negative distances because heap returns smallest member
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
437 h = [(-dist[node], node)]
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
438 seen = {}
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
439 earliest = self.count()
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
440 while h:
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
441 d, n = heapq.heappop(h)
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
442 if n not in seen:
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
443 seen[n] = 1
381
024ee0f8722a Ancestor algorithm fix
mpm@selenic.com
parents: 377
diff changeset
444 r = self.rev(n)
1351
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
445 yield (-d, n)
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
446 for p in self.parents(n):
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
447 heapq.heappush(h, (-dist[p], p))
45
f2b2d5daec30 Fix recursion depth trouble with ancestor algorithm
mpm@selenic.com
parents: 41
diff changeset
448
1351
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
449 def generations(node):
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
450 sg, s = None, {}
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
451 for g,n in ancestors(node):
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
452 if g != sg:
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
453 if sg:
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
454 yield sg, s
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
455 sg, s = g, {n:1}
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
456 else:
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
457 s[n] = 1
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
458 yield sg, s
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
459
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
460 x = generations(a)
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
461 y = generations(b)
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
462 gx = x.next()
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
463 gy = y.next()
45
f2b2d5daec30 Fix recursion depth trouble with ancestor algorithm
mpm@selenic.com
parents: 41
diff changeset
464
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
465 # increment each ancestor list until it is closer to root than
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
466 # the other, or they match
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
467 while 1:
1351
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
468 #print "ancestor gen %s %s" % (gx[0], gy[0])
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
469 if gx[0] == gy[0]:
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
470 # find the intersection
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
471 i = [ n for n in gx[1] if n in gy[1] ]
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
472 if i:
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
473 return i[0]
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
474 else:
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
475 #print "next"
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
476 gy = y.next()
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
477 gx = x.next()
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
478 elif gx[0] < gy[0]:
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
479 #print "next y"
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
480 gy = y.next()
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
481 else:
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
482 #print "next x"
0e2be889ccd7 Repair ancestor logic, fix up test cases
Matt Mackall <mpm@selenic.com>
parents: 1325
diff changeset
483 gx = x.next()
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
484
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
485 def group(self, linkmap):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
486 """calculate a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
487
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
488 Given a list of changeset revs, return a set of deltas and
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
489 metadata corresponding to nodes. the first delta is
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
490 parent(nodes[0]) -> nodes[0] the receiver is guaranteed to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
491 have this parent as it has all history before these
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
492 changesets. parent is parent[0]
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
493 """
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
494 revs = []
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
495 needed = {}
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
496
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
497 # find file nodes/revs that match changeset revs
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
498 for i in xrange(0, self.count()):
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
499 if self.index[i][3] in linkmap:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
500 revs.append(i)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
501 needed[i] = 1
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
502
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
503 # if we don't have any revisions touched by these changesets, bail
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
504 if not revs:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
505 yield struct.pack(">l", 0)
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
506 return
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
507
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
508 # add the parent of the first rev
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
509 p = self.parents(self.node(revs[0]))[0]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
510 revs.insert(0, self.rev(p))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
511
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
512 # for each delta that isn't contiguous in the log, we need to
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
513 # reconstruct the base, reconstruct the result, and then
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
514 # calculate the delta. We also need to do this where we've
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
515 # stored a full version and not a delta
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
516 for i in xrange(0, len(revs) - 1):
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
517 a, b = revs[i], revs[i + 1]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
518 if a + 1 != b or self.base(b) == b:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
519 for j in xrange(self.base(a), a + 1):
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
520 needed[j] = 1
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
521 for j in xrange(self.base(b), b + 1):
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
522 needed[j] = 1
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
523
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
524 # calculate spans to retrieve from datafile
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
525 needed = needed.keys()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
526 needed.sort()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
527 spans = []
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
528 oo = -1
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
529 ol = 0
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
530 for n in needed:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
531 if n < 0: continue
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
532 o = self.start(n)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
533 l = self.length(n)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
534 if oo + ol == o: # can we merge with the previous?
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
535 nl = spans[-1][2]
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
536 nl.append((n, l))
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
537 ol += l
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
538 spans[-1] = (oo, ol, nl)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
539 else:
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
540 oo = o
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
541 ol = l
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
542 spans.append((oo, ol, [(n, l)]))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
543
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
544 # read spans in, divide up chunks
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
545 chunks = {}
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
546 for span in spans:
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
547 # we reopen the file for each span to make http happy for now
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
548 f = self.opener(self.datafile)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
549 f.seek(span[0])
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
550 data = f.read(span[1])
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
551
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
552 # divide up the span
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
553 pos = 0
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
554 for r, l in span[2]:
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
555 chunks[r] = decompress(data[pos: pos + l])
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
556 pos += l
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
557
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
558 # helper to reconstruct intermediate versions
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
559 def construct(text, base, rev):
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
560 bins = [chunks[r] for r in xrange(base + 1, rev + 1)]
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
561 return mdiff.patches(text, bins)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
562
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
563 # build deltas
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
564 deltas = []
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
565 for d in xrange(0, len(revs) - 1):
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
566 a, b = revs[d], revs[d + 1]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
567 n = self.node(b)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
568
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
569 # do we need to construct a new delta?
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
570 if a + 1 != b or self.base(b) == b:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
571 if a >= 0:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
572 base = self.base(a)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
573 ta = chunks[self.base(a)]
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
574 ta = construct(ta, base, a)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
575 else:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
576 ta = ""
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
577
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
578 base = self.base(b)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
579 if a > base:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
580 base = a
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
581 tb = ta
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
582 else:
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
583 tb = chunks[self.base(b)]
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
584 tb = construct(tb, base, b)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
585 d = self.diff(ta, tb)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
586 else:
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
587 d = chunks[b]
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
588
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
589 p = self.parents(n)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
590 meta = n + p[0] + p[1] + linkmap[self.linkrev(n)]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
591 l = struct.pack(">l", len(meta) + len(d) + 4)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
592 yield l
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
593 yield meta
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
594 yield d
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
595
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
596 yield struct.pack(">l", 0)
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
597
1062
6d5a62a549fa pep-0008 cleanup
benoit.boissinot@ens-lyon.fr
parents: 902
diff changeset
598 def addgroup(self, revs, linkmapper, transaction, unique=0):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
599 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
600 add a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
601
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
602 given a set of deltas, add them to the revision log. the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
603 first delta is against its parent, which should be in our
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
604 log, the rest are against the previous delta.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
605 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
606
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
607 #track the base of the current delta log
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
608 r = self.count()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
609 t = r - 1
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
610 node = nullid
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
611
655
b3bba126b04a Fix out of range regression
Matt Mackall <mpm@selenic.com>
parents: 653
diff changeset
612 base = prev = -1
653
94cdd02792b5 Fix corruption resulting from skipping parts of a revision group
Matt Mackall <mpm@selenic.com>
parents: 651
diff changeset
613 start = end = measure = 0
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
614 if r:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
615 start = self.start(self.base(t))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
616 end = self.end(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
617 measure = self.length(self.base(t))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
618 base = self.base(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
619 prev = self.tip()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
620
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
621 transaction.add(self.datafile, end)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
622 transaction.add(self.indexfile, r * struct.calcsize(indexformat))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
623 dfh = self.opener(self.datafile, "a")
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
624 ifh = self.opener(self.indexfile, "a")
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
625
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
626 # loop through our set of deltas
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
627 chain = None
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
628 for chunk in revs:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
629 node, p1, p2, cs = struct.unpack("20s20s20s20s", chunk[:80])
94
7daef883134f Refactor merge code
mpm@selenic.com
parents: 86
diff changeset
630 link = linkmapper(cs)
77
bed15e766511 Fix bug in lazymap code
mpm@selenic.com
parents: 76
diff changeset
631 if node in self.nodemap:
224
ccbcc4d76f81 fix bad assumption about uniqueness of file versions
mpm@selenic.com
parents: 221
diff changeset
632 # this can happen if two branches make the same change
1218
cde6818e082a Add preliminary support for the bundle and unbundle commands
mpm@selenic.com
parents: 1214
diff changeset
633 # if unique:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
634 # raise RevlogError(_("already have %s") % hex(node[:4]))
653
94cdd02792b5 Fix corruption resulting from skipping parts of a revision group
Matt Mackall <mpm@selenic.com>
parents: 651
diff changeset
635 chain = node
224
ccbcc4d76f81 fix bad assumption about uniqueness of file versions
mpm@selenic.com
parents: 221
diff changeset
636 continue
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
637 delta = chunk[80:]
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
638
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
639 if not chain:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
640 # retrieve the parent revision of the delta chain
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
641 chain = p1
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
642 if not chain in self.nodemap:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
643 raise RevlogError(_("unknown base %s") % short(chain[:4]))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
644
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
645 # full versions are inserted when the needed deltas become
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
646 # comparable to the uncompressed text or when the previous
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
647 # version is not the one we have a delta against. We use
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
648 # the size of the previous full rev as a proxy for the
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
649 # current size.
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
650
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
651 if chain == prev:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
652 cdelta = compress(delta)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
653
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
654 if chain != prev or (end - start + len(cdelta)) > measure * 2:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
655 # flush our writes here so we can read it in revision
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
656 dfh.flush()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
657 ifh.flush()
65
d40cc5aacc31 Fix up a bunch of bugs in the new merge code
mpm@selenic.com
parents: 64
diff changeset
658 text = self.revision(chain)
73
ee1cbe841e01 Change revlog to use new patch code
mpm@selenic.com
parents: 71
diff changeset
659 text = self.patches(text, [delta])
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
660 chk = self.addrevision(text, transaction, link, p1, p2)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
661 if chk != node:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
662 raise RevlogError(_("consistency error adding group"))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
663 measure = len(text)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
664 else:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
665 e = (end, len(cdelta), self.base(t), link, p1, p2, node)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
666 self.index.append(e)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
667 self.nodemap[node] = r
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
668 dfh.write(cdelta)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
669 ifh.write(struct.pack(indexformat, *e))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
670
65
d40cc5aacc31 Fix up a bunch of bugs in the new merge code
mpm@selenic.com
parents: 64
diff changeset
671 t, r, chain, prev = r, r + 1, node, node
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
672 start = self.start(self.base(t))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
673 end = self.end(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
674
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
675 dfh.close()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
676 ifh.close()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
677 return node