annotate mercurial/revlog.py @ 1322:b3d44e9b3092

Make revlog constructor more discerning in its treatment of errors.
author Bryan O'Sullivan <bos@serpentine.com>
date Thu, 22 Sep 2005 23:31:44 -0700
parents eb3cc5e2eb89
children 57220daf40e9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
1 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
2 revlog.py - storage back-end for mercurial
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
3
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
4 This provides efficient delta storage with O(1) retrieve and append
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
5 and O(changes) merge between branches
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
6
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
7 Copyright 2005 Matt Mackall <mpm@selenic.com>
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
8
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
9 This software may be used and distributed according to the terms
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
10 of the GNU General Public License, incorporated herein by reference.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
11 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
12
1089
142b5d5ec9cc Break apart hg.py
mpm@selenic.com
parents: 1083
diff changeset
13 from node import *
1322
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
14 from demandload import demandload
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
15 demandload(globals(), "binascii errno heapq mdiff sha struct urllib2 zlib")
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
16
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
17 def hash(text, p1, p2):
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
18 """generate a hash from the given text and its parent hashes
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
19
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
20 This hash combines both the current file contents and its history
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
21 in a manner that makes it easy to distinguish nodes with the same
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
22 content in the revision graph.
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
23 """
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
24 l = [p1, p2]
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
25 l.sort()
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
26 s = sha.new(l[0])
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
27 s.update(l[1])
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
28 s.update(text)
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
29 return s.digest()
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
30
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
31 def compress(text):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
32 """ generate a possibly-compressed representation of text """
112
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
33 if not text: return text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
34 if len(text) < 44:
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
35 if text[0] == '\0': return text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
36 return 'u' + text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
37 bin = zlib.compress(text)
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
38 if len(bin) > len(text):
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
39 if text[0] == '\0': return text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
40 return 'u' + text
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
41 return bin
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
42
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
43 def decompress(bin):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
44 """ decompress the given input """
112
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
45 if not bin: return bin
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
46 t = bin[0]
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
47 if t == '\0': return bin
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
48 if t == 'x': return zlib.decompress(bin)
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
49 if t == 'u': return bin[1:]
1073
7b35a980b982 [PATCH] raise exceptions with Exception subclasses
Bart Trojanowski <bart@jukie.net>
parents: 1062
diff changeset
50 raise RevlogError("unknown compression type %s" % t)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
51
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
52 indexformat = ">4l20s20s20s"
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
53
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
54 class lazyparser:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
55 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
56 this class avoids the need to parse the entirety of large indices
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
57
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
58 By default we parse and load 1000 entries at a time.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
59
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
60 If no position is specified, we load the whole index, and replace
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
61 the lazy objects in revlog with the underlying objects for
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
62 efficiency in cases where we look at most of the nodes.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
63 """
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
64 def __init__(self, data, revlog):
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
65 self.data = data
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
66 self.s = struct.calcsize(indexformat)
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
67 self.l = len(data)/self.s
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
68 self.index = [None] * self.l
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
69 self.map = {nullid: -1}
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
70 self.all = 0
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
71 self.revlog = revlog
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
72
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
73 def load(self, pos=None):
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
74 if self.all: return
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
75 if pos is not None:
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
76 block = pos / 1000
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
77 i = block * 1000
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
78 end = min(self.l, i + 1000)
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
79 else:
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
80 self.all = 1
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
81 i = 0
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
82 end = self.l
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
83 self.revlog.index = self.index
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
84 self.revlog.nodemap = self.map
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
85
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
86 while i < end:
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
87 d = self.data[i * self.s: (i + 1) * self.s]
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
88 e = struct.unpack(indexformat, d)
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
89 self.index[i] = e
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
90 self.map[e[6]] = i
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
91 i += 1
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
92
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
93 class lazyindex:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
94 """a lazy version of the index array"""
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
95 def __init__(self, parser):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
96 self.p = parser
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
97 def __len__(self):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
98 return len(self.p.index)
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
99 def load(self, pos):
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
100 self.p.load(pos)
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
101 return self.p.index[pos]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
102 def __getitem__(self, pos):
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
103 return self.p.index[pos] or self.load(pos)
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
104 def append(self, e):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
105 self.p.index.append(e)
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
106
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
107 class lazymap:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
108 """a lazy version of the node map"""
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
109 def __init__(self, parser):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
110 self.p = parser
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
111 def load(self, key):
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
112 if self.p.all: return
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
113 n = self.p.data.find(key)
1214
34706a835d4a Smarter handling of revlog key errors
mpm@selenic.com
parents: 1201
diff changeset
114 if n < 0:
34706a835d4a Smarter handling of revlog key errors
mpm@selenic.com
parents: 1201
diff changeset
115 raise KeyError(key)
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
116 pos = n / self.p.s
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
117 self.p.load(pos)
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
118 def __contains__(self, key):
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
119 self.p.load()
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
120 return key in self.p.map
97
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
121 def __iter__(self):
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
122 yield nullid
97
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
123 for i in xrange(self.p.l):
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
124 try:
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
125 yield self.p.index[i][6]
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
126 except:
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
127 self.p.load(i)
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
128 yield self.p.index[i][6]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
129 def __getitem__(self, key):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
130 try:
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
131 return self.p.map[key]
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
132 except KeyError:
86
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
133 try:
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
134 self.load(key)
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
135 return self.p.map[key]
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
136 except KeyError:
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
137 raise KeyError("node " + hex(key))
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
138 def __setitem__(self, key, val):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
139 self.p.map[key] = val
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
140
1073
7b35a980b982 [PATCH] raise exceptions with Exception subclasses
Bart Trojanowski <bart@jukie.net>
parents: 1062
diff changeset
141 class RevlogError(Exception): pass
7b35a980b982 [PATCH] raise exceptions with Exception subclasses
Bart Trojanowski <bart@jukie.net>
parents: 1062
diff changeset
142
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
143 class revlog:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
144 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
145 the underlying revision storage object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
146
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
147 A revlog consists of two parts, an index and the revision data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
148
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
149 The index is a file with a fixed record size containing
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
150 information on each revision, includings its nodeid (hash), the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
151 nodeids of its parents, the position and offset of its data within
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
152 the data file, and the revision it's based on. Finally, each entry
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
153 contains a linkrev entry that can serve as a pointer to external
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
154 data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
155
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
156 The revision data itself is a linear collection of data chunks.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
157 Each chunk represents a revision and is usually represented as a
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
158 delta against the previous chunk. To bound lookup time, runs of
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
159 deltas are limited to about 2 times the length of the original
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
160 version data. This makes retrieval of a version proportional to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
161 its size, or O(1) relative to the number of revisions.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
162
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
163 Both pieces of the revlog are written to in an append-only
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
164 fashion, which means we never need to rewrite a file to insert or
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
165 remove data, and can use some simple techniques to avoid the need
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
166 for locking while reading.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
167 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
168 def __init__(self, opener, indexfile, datafile):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
169 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
170 create a revlog object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
171
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
172 opener is a function that abstracts the file opening operation
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
173 and can be used to implement COW semantics or the like.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
174 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
175 self.indexfile = indexfile
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
176 self.datafile = datafile
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
177 self.opener = opener
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
178 self.cache = None
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
179
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
180 try:
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
181 i = self.opener(self.indexfile).read()
1322
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
182 except urllib2.URLError:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
183 raise
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
184 except IOError, inst:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
185 if inst.errno != errno.ENOENT:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
186 raise
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
187 i = ""
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
188
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
189 if len(i) > 10000:
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
190 # big index, let's parse it on demand
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
191 parser = lazyparser(i, self)
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
192 self.index = lazyindex(parser)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
193 self.nodemap = lazymap(parser)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
194 else:
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
195 s = struct.calcsize(indexformat)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
196 l = len(i) / s
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
197 self.index = [None] * l
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
198 m = [None] * l
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
199
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
200 n = 0
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
201 for f in xrange(0, len(i), s):
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
202 # offset, size, base, linkrev, p1, p2, nodeid
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
203 e = struct.unpack(indexformat, i[f:f + s])
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
204 m[n] = (e[6], n)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
205 self.index[n] = e
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
206 n += 1
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
207
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
208 self.nodemap = dict(m)
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
209 self.nodemap[nullid] = -1
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
210
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
211 def tip(self): return self.node(len(self.index) - 1)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
212 def count(self): return len(self.index)
26
9cf83bf9ad38 Simplify integrity checking
mpm@selenic.com
parents: 14
diff changeset
213 def node(self, rev): return (rev < 0) and nullid or self.index[rev][6]
1201
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
214 def rev(self, node):
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
215 try:
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
216 return self.nodemap[node]
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
217 except KeyError:
1214
34706a835d4a Smarter handling of revlog key errors
mpm@selenic.com
parents: 1201
diff changeset
218 raise RevlogError('%s: no node %s' % (self.indexfile, hex(node)))
1201
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
219 def linkrev(self, node): return self.index[self.rev(node)][3]
2
ecf3fd948051 Handle nullid better for ancestor
mpm@selenic.com
parents: 0
diff changeset
220 def parents(self, node):
ecf3fd948051 Handle nullid better for ancestor
mpm@selenic.com
parents: 0
diff changeset
221 if node == nullid: return (nullid, nullid)
1201
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
222 return self.index[self.rev(node)][4:6]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
223
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
224 def start(self, rev): return self.index[rev][0]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
225 def length(self, rev): return self.index[rev][1]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
226 def end(self, rev): return self.start(rev) + self.length(rev)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
227 def base(self, rev): return self.index[rev][2]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
228
1074
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
229 def reachable(self, rev, stop=None):
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
230 reachable = {}
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
231 visit = [rev]
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
232 reachable[rev] = 1
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
233 if stop:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
234 stopn = self.rev(stop)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
235 else:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
236 stopn = 0
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
237 while visit:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
238 n = visit.pop(0)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
239 if n == stop:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
240 continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
241 if n == nullid:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
242 continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
243 for p in self.parents(n):
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
244 if self.rev(p) < stopn:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
245 continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
246 if p not in reachable:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
247 reachable[p] = 1
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
248 visit.append(p)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
249 return reachable
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
250
902
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
251 def heads(self, stop=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
252 """return the list of all nodes that have no children"""
221
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
253 p = {}
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
254 h = []
902
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
255 stoprev = 0
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
256 if stop and stop in self.nodemap:
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
257 stoprev = self.rev(stop)
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
258
243
9a9ea2d1d3c4 fix heads for rev 0
mpm@selenic.com
parents: 241
diff changeset
259 for r in range(self.count() - 1, -1, -1):
221
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
260 n = self.node(r)
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
261 if n not in p:
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
262 h.append(n)
902
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
263 if n == stop:
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
264 break
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
265 if r < stoprev:
c749ca37aed1 Add optional stop revision to revlog.heads
mason@suse.com
parents: 896
diff changeset
266 break
221
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
267 for pn in self.parents(n):
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
268 p[pn] = 1
2bfe525ef6ca Beginning of multi-head support
mpm@selenic.com
parents: 208
diff changeset
269 return h
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
270
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
271 def children(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
272 """find the children of a given node"""
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
273 c = []
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
274 p = self.rev(node)
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
275 for r in range(p + 1, self.count()):
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
276 n = self.node(r)
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
277 for pn in self.parents(n):
854
473c030d34a6 Fixed revlog.children.
Tristan Wibberley <tristan@wibberley.org>
parents: 655
diff changeset
278 if pn == node:
473c030d34a6 Fixed revlog.children.
Tristan Wibberley <tristan@wibberley.org>
parents: 655
diff changeset
279 c.append(n)
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
280 continue
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
281 elif pn == nullid:
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
282 continue
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
283 return c
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
284
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
285 def lookup(self, id):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
286 """locate a node based on revision number or subset of hex nodeid"""
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
287 try:
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
288 rev = int(id)
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
289 if str(rev) != id: raise ValueError
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
290 if rev < 0: rev = self.count() + rev
476
0a338d506268 Really _call_ method revlog.count in revlog.lookup()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 469
diff changeset
291 if rev < 0 or rev >= self.count(): raise ValueError
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
292 return self.node(rev)
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
293 except (ValueError, OverflowError):
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
294 c = []
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
295 for n in self.nodemap:
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
296 if hex(n).startswith(id):
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
297 c.append(n)
1232
eb3cc5e2eb89 Revert some exception type changes in revlog
mpm@selenic.com
parents: 1218
diff changeset
298 if len(c) > 1: raise KeyError("Ambiguous identifier")
eb3cc5e2eb89 Revert some exception type changes in revlog
mpm@selenic.com
parents: 1218
diff changeset
299 if len(c) < 1: raise KeyError("No match found")
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
300 return c[0]
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
301
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
302 return None
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
303
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
304 def diff(self, a, b):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
305 """return a delta between two revisions"""
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
306 return mdiff.textdiff(a, b)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
307
73
ee1cbe841e01 Change revlog to use new patch code
mpm@selenic.com
parents: 71
diff changeset
308 def patches(self, t, pl):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
309 """apply a list of patches to a string"""
73
ee1cbe841e01 Change revlog to use new patch code
mpm@selenic.com
parents: 71
diff changeset
310 return mdiff.patches(t, pl)
ee1cbe841e01 Change revlog to use new patch code
mpm@selenic.com
parents: 71
diff changeset
311
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
312 def delta(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
313 """return or calculate a delta between a node and its predecessor"""
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
314 r = self.rev(node)
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
315 b = self.base(r)
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
316 if r == b:
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
317 return self.diff(self.revision(self.node(r - 1)),
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
318 self.revision(node))
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
319 else:
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
320 f = self.opener(self.datafile)
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
321 f.seek(self.start(r))
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
322 data = f.read(self.length(r))
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
323 return decompress(data)
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
324
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
325 def revision(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
326 """return an uncompressed revision of a given"""
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
327 if node == nullid: return ""
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
328 if self.cache and self.cache[0] == node: return self.cache[2]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
329
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
330 # look up what we need to read
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
331 text = None
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
332 rev = self.rev(node)
117
2ac722ad1a9d Make revision code slightly faster
mpm@selenic.com
parents: 116
diff changeset
333 start, length, base, link, p1, p2, node = self.index[rev]
2ac722ad1a9d Make revision code slightly faster
mpm@selenic.com
parents: 116
diff changeset
334 end = start + length
2ac722ad1a9d Make revision code slightly faster
mpm@selenic.com
parents: 116
diff changeset
335 if base != rev: start = self.start(base)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
336
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
337 # do we have useful data cached?
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
338 if self.cache and self.cache[1] >= base and self.cache[1] < rev:
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
339 base = self.cache[1]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
340 start = self.start(base + 1)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
341 text = self.cache[2]
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
342 last = 0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
343
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
344 f = self.opener(self.datafile)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
345 f.seek(start)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
346 data = f.read(end - start)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
347
651
08b65d66f3e7 Fix an odd revlog bug
Matt Mackall <mpm@selenic.com>
parents: 644
diff changeset
348 if text is None:
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
349 last = self.length(base)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
350 text = decompress(data[:last])
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
351
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
352 bins = []
64
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
353 for r in xrange(base + 1, rev + 1):
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
354 s = self.length(r)
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
355 bins.append(decompress(data[last:last + s]))
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
356 last = last + s
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
357
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
358 text = mdiff.patches(text, bins)
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
359
26
9cf83bf9ad38 Simplify integrity checking
mpm@selenic.com
parents: 14
diff changeset
360 if node != hash(text, p1, p2):
1214
34706a835d4a Smarter handling of revlog key errors
mpm@selenic.com
parents: 1201
diff changeset
361 raise RevlogError("integrity check failed on %s:%d"
98
3dde7c87e36d Add paranoia to diff code
mpm@selenic.com
parents: 97
diff changeset
362 % (self.datafile, rev))
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
363
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
364 self.cache = (node, rev, text)
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
365 return text
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
366
644
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
367 def addrevision(self, text, transaction, link, p1=None, p2=None, d=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
368 """add a revision to the log
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
369
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
370 text - the revision data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
371 transaction - the transaction object used for rollback
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
372 link - the linkrev data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
373 p1, p2 - the parent nodeids of the revision
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
374 d - an optional precomputed delta
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
375 """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
376 if text is None: text = ""
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
377 if p1 is None: p1 = self.tip()
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
378 if p2 is None: p2 = nullid
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
379
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
380 node = hash(text, p1, p2)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
381
301
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
382 if node in self.nodemap:
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
383 return node
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
384
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
385 n = self.count()
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
386 t = n - 1
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
387
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
388 if n:
64
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
389 base = self.base(t)
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
390 start = self.start(base)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
391 end = self.end(t)
644
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
392 if not d:
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
393 prev = self.revision(self.tip())
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
394 d = self.diff(prev, text)
98
3dde7c87e36d Add paranoia to diff code
mpm@selenic.com
parents: 97
diff changeset
395 data = compress(d)
64
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
396 dist = end - start + len(data)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
397
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
398 # full versions are inserted when the needed deltas
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
399 # become comparable to the uncompressed text
64
b3e2ddff0159 Diff in subdirectories from Jake Edge
mpm@selenic.com
parents: 46
diff changeset
400 if not n or dist > len(text) * 2:
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
401 data = compress(text)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
402 base = n
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
403 else:
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
404 base = self.base(t)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
405
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
406 offset = 0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
407 if t >= 0:
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
408 offset = self.end(t)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
409
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
410 e = (offset, len(data), base, link, p1, p2, node)
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
411
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
412 self.index.append(e)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
413 self.nodemap[node] = n
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
414 entry = struct.pack(indexformat, *e)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
415
26
9cf83bf9ad38 Simplify integrity checking
mpm@selenic.com
parents: 14
diff changeset
416 transaction.add(self.datafile, e[0])
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
417 self.opener(self.datafile, "a").write(data)
41
df3f46253878 Fix truncate logic for indices again
mpm@selenic.com
parents: 36
diff changeset
418 transaction.add(self.indexfile, n * len(entry))
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
419 self.opener(self.indexfile, "a").write(entry)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
420
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
421 self.cache = (node, n, text)
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
422 return node
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
423
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
424 def ancestor(self, a, b):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
425 """calculate the least common ancestor of nodes a and b"""
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
426 # calculate the distance of every node from root
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
427 dist = {nullid: 0}
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
428 for i in xrange(self.count()):
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
429 n = self.node(i)
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
430 p1, p2 = self.parents(n)
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
431 dist[n] = max(dist[p1], dist[p2]) + 1
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
432
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
433 # traverse ancestors in order of decreasing distance from root
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
434 def ancestors(node):
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
435 # we store negative distances because heap returns smallest member
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
436 h = [(-dist[node], node)]
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
437 seen = {}
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
438 earliest = self.count()
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
439 while h:
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
440 d, n = heapq.heappop(h)
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
441 if n not in seen:
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
442 seen[n] = 1
381
024ee0f8722a Ancestor algorithm fix
mpm@selenic.com
parents: 377
diff changeset
443 r = self.rev(n)
024ee0f8722a Ancestor algorithm fix
mpm@selenic.com
parents: 377
diff changeset
444 yield (-d, r, n)
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
445 for p in self.parents(n):
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
446 heapq.heappush(h, (-dist[p], p))
45
f2b2d5daec30 Fix recursion depth trouble with ancestor algorithm
mpm@selenic.com
parents: 41
diff changeset
447
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
448 x = ancestors(a)
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
449 y = ancestors(b)
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
450 lx = x.next()
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
451 ly = y.next()
45
f2b2d5daec30 Fix recursion depth trouble with ancestor algorithm
mpm@selenic.com
parents: 41
diff changeset
452
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
453 # increment each ancestor list until it is closer to root than
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
454 # the other, or they match
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
455 while 1:
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
456 if lx == ly:
381
024ee0f8722a Ancestor algorithm fix
mpm@selenic.com
parents: 377
diff changeset
457 return lx[2]
147
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
458 elif lx < ly:
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
459 ly = y.next()
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
460 elif lx > ly:
b6d8ed7aeba0 A new ancestor algorithm
mpm@selenic.com
parents: 126
diff changeset
461 lx = x.next()
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
462
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
463 def group(self, linkmap):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
464 """calculate a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
465
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
466 Given a list of changeset revs, return a set of deltas and
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
467 metadata corresponding to nodes. the first delta is
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
468 parent(nodes[0]) -> nodes[0] the receiver is guaranteed to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
469 have this parent as it has all history before these
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
470 changesets. parent is parent[0]
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
471 """
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
472 revs = []
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
473 needed = {}
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
474
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
475 # find file nodes/revs that match changeset revs
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
476 for i in xrange(0, self.count()):
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
477 if self.index[i][3] in linkmap:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
478 revs.append(i)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
479 needed[i] = 1
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
480
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
481 # if we don't have any revisions touched by these changesets, bail
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
482 if not revs:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
483 yield struct.pack(">l", 0)
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
484 return
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
485
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
486 # add the parent of the first rev
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
487 p = self.parents(self.node(revs[0]))[0]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
488 revs.insert(0, self.rev(p))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
489
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
490 # for each delta that isn't contiguous in the log, we need to
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
491 # reconstruct the base, reconstruct the result, and then
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
492 # calculate the delta. We also need to do this where we've
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
493 # stored a full version and not a delta
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
494 for i in xrange(0, len(revs) - 1):
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
495 a, b = revs[i], revs[i + 1]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
496 if a + 1 != b or self.base(b) == b:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
497 for j in xrange(self.base(a), a + 1):
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
498 needed[j] = 1
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
499 for j in xrange(self.base(b), b + 1):
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
500 needed[j] = 1
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
501
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
502 # calculate spans to retrieve from datafile
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
503 needed = needed.keys()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
504 needed.sort()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
505 spans = []
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
506 oo = -1
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
507 ol = 0
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
508 for n in needed:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
509 if n < 0: continue
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
510 o = self.start(n)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
511 l = self.length(n)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
512 if oo + ol == o: # can we merge with the previous?
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
513 nl = spans[-1][2]
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
514 nl.append((n, l))
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
515 ol += l
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
516 spans[-1] = (oo, ol, nl)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
517 else:
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
518 oo = o
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
519 ol = l
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
520 spans.append((oo, ol, [(n, l)]))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
521
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
522 # read spans in, divide up chunks
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
523 chunks = {}
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
524 for span in spans:
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
525 # we reopen the file for each span to make http happy for now
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
526 f = self.opener(self.datafile)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
527 f.seek(span[0])
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
528 data = f.read(span[1])
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
529
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
530 # divide up the span
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
531 pos = 0
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
532 for r, l in span[2]:
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
533 chunks[r] = decompress(data[pos: pos + l])
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
534 pos += l
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
535
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
536 # helper to reconstruct intermediate versions
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
537 def construct(text, base, rev):
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
538 bins = [chunks[r] for r in xrange(base + 1, rev + 1)]
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
539 return mdiff.patches(text, bins)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
540
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
541 # build deltas
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
542 deltas = []
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
543 for d in xrange(0, len(revs) - 1):
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
544 a, b = revs[d], revs[d + 1]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
545 n = self.node(b)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
546
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
547 # do we need to construct a new delta?
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
548 if a + 1 != b or self.base(b) == b:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
549 if a >= 0:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
550 base = self.base(a)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
551 ta = chunks[self.base(a)]
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
552 ta = construct(ta, base, a)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
553 else:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
554 ta = ""
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
555
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
556 base = self.base(b)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
557 if a > base:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
558 base = a
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
559 tb = ta
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
560 else:
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
561 tb = chunks[self.base(b)]
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
562 tb = construct(tb, base, b)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
563 d = self.diff(ta, tb)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
564 else:
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
565 d = chunks[b]
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
566
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
567 p = self.parents(n)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
568 meta = n + p[0] + p[1] + linkmap[self.linkrev(n)]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
569 l = struct.pack(">l", len(meta) + len(d) + 4)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
570 yield l
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
571 yield meta
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
572 yield d
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
573
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
574 yield struct.pack(">l", 0)
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
575
1062
6d5a62a549fa pep-0008 cleanup
benoit.boissinot@ens-lyon.fr
parents: 902
diff changeset
576 def addgroup(self, revs, linkmapper, transaction, unique=0):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
577 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
578 add a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
579
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
580 given a set of deltas, add them to the revision log. the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
581 first delta is against its parent, which should be in our
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
582 log, the rest are against the previous delta.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
583 """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
584
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
585 #track the base of the current delta log
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
586 r = self.count()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
587 t = r - 1
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
588 node = nullid
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
589
655
b3bba126b04a Fix out of range regression
Matt Mackall <mpm@selenic.com>
parents: 653
diff changeset
590 base = prev = -1
653
94cdd02792b5 Fix corruption resulting from skipping parts of a revision group
Matt Mackall <mpm@selenic.com>
parents: 651
diff changeset
591 start = end = measure = 0
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
592 if r:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
593 start = self.start(self.base(t))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
594 end = self.end(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
595 measure = self.length(self.base(t))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
596 base = self.base(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
597 prev = self.tip()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
598
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
599 transaction.add(self.datafile, end)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
600 transaction.add(self.indexfile, r * struct.calcsize(indexformat))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
601 dfh = self.opener(self.datafile, "a")
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
602 ifh = self.opener(self.indexfile, "a")
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
603
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
604 # loop through our set of deltas
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
605 chain = None
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
606 for chunk in revs:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
607 node, p1, p2, cs = struct.unpack("20s20s20s20s", chunk[:80])
94
7daef883134f Refactor merge code
mpm@selenic.com
parents: 86
diff changeset
608 link = linkmapper(cs)
77
bed15e766511 Fix bug in lazymap code
mpm@selenic.com
parents: 76
diff changeset
609 if node in self.nodemap:
224
ccbcc4d76f81 fix bad assumption about uniqueness of file versions
mpm@selenic.com
parents: 221
diff changeset
610 # this can happen if two branches make the same change
1218
cde6818e082a Add preliminary support for the bundle and unbundle commands
mpm@selenic.com
parents: 1214
diff changeset
611 # if unique:
cde6818e082a Add preliminary support for the bundle and unbundle commands
mpm@selenic.com
parents: 1214
diff changeset
612 # raise RevlogError("already have %s" % hex(node[:4]))
653
94cdd02792b5 Fix corruption resulting from skipping parts of a revision group
Matt Mackall <mpm@selenic.com>
parents: 651
diff changeset
613 chain = node
224
ccbcc4d76f81 fix bad assumption about uniqueness of file versions
mpm@selenic.com
parents: 221
diff changeset
614 continue
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
615 delta = chunk[80:]
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
616
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
617 if not chain:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
618 # retrieve the parent revision of the delta chain
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
619 chain = p1
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
620 if not chain in self.nodemap:
1073
7b35a980b982 [PATCH] raise exceptions with Exception subclasses
Bart Trojanowski <bart@jukie.net>
parents: 1062
diff changeset
621 raise RevlogError("unknown base %s" % short(chain[:4]))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
622
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
623 # full versions are inserted when the needed deltas become
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
624 # comparable to the uncompressed text or when the previous
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
625 # version is not the one we have a delta against. We use
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
626 # the size of the previous full rev as a proxy for the
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
627 # current size.
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
628
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
629 if chain == prev:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
630 cdelta = compress(delta)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
631
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
632 if chain != prev or (end - start + len(cdelta)) > measure * 2:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
633 # flush our writes here so we can read it in revision
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
634 dfh.flush()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
635 ifh.flush()
65
d40cc5aacc31 Fix up a bunch of bugs in the new merge code
mpm@selenic.com
parents: 64
diff changeset
636 text = self.revision(chain)
73
ee1cbe841e01 Change revlog to use new patch code
mpm@selenic.com
parents: 71
diff changeset
637 text = self.patches(text, [delta])
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
638 chk = self.addrevision(text, transaction, link, p1, p2)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
639 if chk != node:
1073
7b35a980b982 [PATCH] raise exceptions with Exception subclasses
Bart Trojanowski <bart@jukie.net>
parents: 1062
diff changeset
640 raise RevlogError("consistency error adding group")
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
641 measure = len(text)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
642 else:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
643 e = (end, len(cdelta), self.base(t), link, p1, p2, node)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
644 self.index.append(e)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
645 self.nodemap[node] = r
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
646 dfh.write(cdelta)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
647 ifh.write(struct.pack(indexformat, *e))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
648
65
d40cc5aacc31 Fix up a bunch of bugs in the new merge code
mpm@selenic.com
parents: 64
diff changeset
649 t, r, chain, prev = r, r + 1, node, node
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
650 start = self.start(self.base(t))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
651 end = self.end(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
652
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
653 dfh.close()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
654 ifh.close()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
655 return node