ethereum.forks.gray_glacier.trie
State Trie.
.. contents:: Table of Contents :backlinks: none :local:
Introduction
The state trie is the structure responsible for storing
.fork_types.Account objects.
EMPTY_TRIE_ROOT
| 58 | EMPTY_TRIE_ROOT = Root( | 
|---|---|
| 59 |     hex_to_bytes( | 
| 60 |         "56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421" | 
| 61 |     ) | 
| 62 | ) | 
Node
| 64 | Node = Account | Bytes | LegacyTransaction | Receipt | Uint | U256 | None | 
|---|
K
| 65 | K = TypeVar("K", bound=Bytes) | 
|---|
V
| 66 | V = TypeVar( | 
|---|---|
| 67 |     "V", | 
| 68 |     Optional[Account], | 
| 69 |     Optional[Bytes], | 
| 70 |     Bytes, | 
| 71 |     Optional[LegacyTransaction | Bytes], | 
| 72 |     Optional[Receipt | Bytes], | 
| 73 |     Uint, | 
| 74 |     U256, | 
| 75 | ) | 
LeafNode
Leaf node in the Merkle Trie.
| 78 | @slotted_freezable | 
|---|
| 79 | @dataclass | 
|---|
class LeafNode:
rest_of_key
| 83 |     rest_of_key: Bytes | 
|---|
value
| 84 |     value: Extended | 
|---|
ExtensionNode
Extension node in the Merkle Trie.
| 87 | @slotted_freezable | 
|---|
| 88 | @dataclass | 
|---|
class ExtensionNode:
key_segment
| 92 |     key_segment: Bytes | 
|---|
subnode
| 93 |     subnode: Extended | 
|---|
BranchSubnodes
| 96 | BranchSubnodes = Tuple[ | 
|---|---|
| 97 |     Extended, | 
| 98 |     Extended, | 
| 99 |     Extended, | 
| 100 |     Extended, | 
| 101 |     Extended, | 
| 102 |     Extended, | 
| 103 |     Extended, | 
| 104 |     Extended, | 
| 105 |     Extended, | 
| 106 |     Extended, | 
| 107 |     Extended, | 
| 108 |     Extended, | 
| 109 |     Extended, | 
| 110 |     Extended, | 
| 111 |     Extended, | 
| 112 |     Extended, | 
| 113 | ] | 
BranchNode
Branch node in the Merkle Trie.
| 116 | @slotted_freezable | 
|---|
| 117 | @dataclass | 
|---|
class BranchNode:
subnodes
| 121 |     subnodes: BranchSubnodes | 
|---|
value
| 122 |     value: Extended | 
|---|
InternalNode
| 125 | InternalNode = LeafNode | ExtensionNode | BranchNode | 
|---|
encode_internal_node
Encodes a Merkle Trie node into its RLP form. The RLP will then be
serialized into a Bytes and hashed unless it is less that 32 bytes
when serialized.
This function also accepts None, representing the absence of a node,
which is encoded to b"".
Parameters
node : Optional[InternalNode] The node to encode.
Returns
encoded : rlp.RLP
The node encoded as RLP.
                def encode_internal_node(node: Optional[InternalNode]) -> Extended:
            
| 129 |     """ | 
|---|---|
| 130 |     Encodes a Merkle Trie node into its RLP form. The RLP will then be | 
| 131 |     serialized into a `Bytes` and hashed unless it is less that 32 bytes | 
| 132 |     when serialized. | 
| 133 |  | 
| 134 |     This function also accepts `None`, representing the absence of a node, | 
| 135 |     which is encoded to `b""`. | 
| 136 |  | 
| 137 |     Parameters | 
| 138 |     ---------- | 
| 139 |     node : Optional[InternalNode] | 
| 140 |         The node to encode. | 
| 141 |  | 
| 142 |     Returns | 
| 143 |     ------- | 
| 144 |     encoded : `rlp.RLP` | 
| 145 |         The node encoded as RLP. | 
| 146 |  | 
| 147 |     """ | 
| 148 |     unencoded: Extended | 
| 149 |     if node is None: | 
| 150 |         unencoded = b"" | 
| 151 |     elif isinstance(node, LeafNode): | 
| 152 |         unencoded = ( | 
| 153 |             nibble_list_to_compact(node.rest_of_key, True), | 
| 154 |             node.value, | 
| 155 |         ) | 
| 156 |     elif isinstance(node, ExtensionNode): | 
| 157 |         unencoded = ( | 
| 158 |             nibble_list_to_compact(node.key_segment, False), | 
| 159 |             node.subnode, | 
| 160 |         ) | 
| 161 |     elif isinstance(node, BranchNode): | 
| 162 |         unencoded = list(node.subnodes) + [node.value] | 
| 163 |     else: | 
| 164 |         raise AssertionError(f"Invalid internal node type {type(node)}!") | 
| 165 | |
| 166 |     encoded = rlp.encode(unencoded) | 
| 167 |     if len(encoded) < 32: | 
| 168 |         return unencoded | 
| 169 |     else: | 
| 170 |         return keccak256(encoded) | 
encode_node
Encode a Node for storage in the Merkle Trie.
Currently mostly an unimplemented stub.
                def encode_node(node: Node, storage_root: Optional[Bytes]) -> Bytes:
            
| 174 |     """ | 
|---|---|
| 175 |     Encode a Node for storage in the Merkle Trie. | 
| 176 |  | 
| 177 |     Currently mostly an unimplemented stub. | 
| 178 |     """ | 
| 179 |     if isinstance(node, Account): | 
| 180 |         assert storage_root is not None | 
| 181 |         return encode_account(node, storage_root) | 
| 182 |     elif isinstance(node, (LegacyTransaction, Receipt, U256)): | 
| 183 |         return rlp.encode(node) | 
| 184 |     elif isinstance(node, Bytes): | 
| 185 |         return node | 
| 186 |     else: | 
| 187 |         return previous_trie.encode_node(node, storage_root) | 
Trie
The Merkle Trie.
| 190 | @dataclass | 
|---|
class Trie:
secured
| 196 |     secured: bool | 
|---|
default
| 197 |     default: V | 
|---|
_data
| 198 |     _data: Dict[K, V] = field(default_factory=dict) | 
|---|
copy_trie
Create a copy of trie. Since only frozen objects may be stored in tries,
the contents are reused.
Parameters
trie: Trie
Trie to copy.
Returns
new_trie : Trie[K, V]
A copy of the trie.
                def copy_trie(trie: Trie[K, V]) -> Trie[K, V]:
            
| 202 |     """ | 
|---|---|
| 203 |     Create a copy of `trie`. Since only frozen objects may be stored in tries, | 
| 204 |     the contents are reused. | 
| 205 |  | 
| 206 |     Parameters | 
| 207 |     ---------- | 
| 208 |     trie: `Trie` | 
| 209 |         Trie to copy. | 
| 210 |  | 
| 211 |     Returns | 
| 212 |     ------- | 
| 213 |     new_trie : `Trie[K, V]` | 
| 214 |         A copy of the trie. | 
| 215 |  | 
| 216 |     """ | 
| 217 |     return Trie(trie.secured, trie.default, copy.copy(trie._data)) | 
trie_set
Stores an item in a Merkle Trie.
This method deletes the key if value == trie.default, because the Merkle
Trie represents the default value by omitting it from the trie.
Parameters
trie: Trie
Trie to store in.
key : Bytes
Key to lookup.
value : V
Node to insert at key.
                def trie_set(trie: Trie[K, V], key: K, value: V) -> None:
            
| 221 |     """ | 
|---|---|
| 222 |     Stores an item in a Merkle Trie. | 
| 223 |  | 
| 224 |     This method deletes the key if `value == trie.default`, because the Merkle | 
| 225 |     Trie represents the default value by omitting it from the trie. | 
| 226 |  | 
| 227 |     Parameters | 
| 228 |     ---------- | 
| 229 |     trie: `Trie` | 
| 230 |         Trie to store in. | 
| 231 |     key : `Bytes` | 
| 232 |         Key to lookup. | 
| 233 |     value : `V` | 
| 234 |         Node to insert at `key`. | 
| 235 |  | 
| 236 |     """ | 
| 237 |     if value == trie.default: | 
| 238 |         if key in trie._data: | 
| 239 |             del trie._data[key] | 
| 240 |     else: | 
| 241 |         trie._data[key] = value | 
trie_get
Gets an item from the Merkle Trie.
This method returns trie.default if the key is missing.
Parameters
trie: Trie to lookup in. key : Key to lookup.
Returns
node : V
Node at key in the trie.
                def trie_get(trie: Trie[K, V], key: K) -> V:
            
| 245 |     """ | 
|---|---|
| 246 |     Gets an item from the Merkle Trie. | 
| 247 |  | 
| 248 |     This method returns `trie.default` if the key is missing. | 
| 249 |  | 
| 250 |     Parameters | 
| 251 |     ---------- | 
| 252 |     trie: | 
| 253 |         Trie to lookup in. | 
| 254 |     key : | 
| 255 |         Key to lookup. | 
| 256 |  | 
| 257 |     Returns | 
| 258 |     ------- | 
| 259 |     node : `V` | 
| 260 |         Node at `key` in the trie. | 
| 261 |  | 
| 262 |     """ | 
| 263 |     return trie._data.get(key, trie.default) | 
common_prefix_length
Find the longest common prefix of two sequences.
                def common_prefix_length(a: Sequence, b: Sequence) -> int:
            
| 267 |     """ | 
|---|---|
| 268 |     Find the longest common prefix of two sequences. | 
| 269 |     """ | 
| 270 |     for i in range(len(a)): | 
| 271 |         if i >= len(b) or a[i] != b[i]: | 
| 272 |             return i | 
| 273 |     return len(a) | 
nibble_list_to_compact
Compresses nibble-list into a standard byte array with a flag.
A nibble-list is a list of byte values no greater than 15. The flag is
encoded in high nibble of the highest byte. The flag nibble can be broken
down into two two-bit flags.
Highest nibble::
+---+---+----------+--------+
| _ | _ | is_leaf | parity |
+---+---+----------+--------+
  3   2      1         0
The lowest bit of the nibble encodes the parity of the length of the
remaining nibbles -- 0 when even and 1 when odd. The second lowest bit
is used to distinguish leaf and extension nodes. The other two bits are not
used.
Parameters
x : Array of nibbles. is_leaf : True if this is part of a leaf node, or false if it is an extension node.
Returns
compressed : bytearray
Compact byte array.
                def nibble_list_to_compact(x: Bytes, is_leaf: bool) -> Bytes:
            
| 277 |     """ | 
|---|---|
| 278 |     Compresses nibble-list into a standard byte array with a flag. | 
| 279 |  | 
| 280 |     A nibble-list is a list of byte values no greater than `15`. The flag is | 
| 281 |     encoded in high nibble of the highest byte. The flag nibble can be broken | 
| 282 |     down into two two-bit flags. | 
| 283 |  | 
| 284 |     Highest nibble:: | 
| 285 |  | 
| 286 |         +---+---+----------+--------+ | 
| 287 |         | _ | _ | is_leaf | parity | | 
| 288 |         +---+---+----------+--------+ | 
| 289 |           3   2      1         0 | 
| 290 |  | 
| 291 |  | 
| 292 |     The lowest bit of the nibble encodes the parity of the length of the | 
| 293 |     remaining nibbles -- `0` when even and `1` when odd. The second lowest bit | 
| 294 |     is used to distinguish leaf and extension nodes. The other two bits are not | 
| 295 |     used. | 
| 296 |  | 
| 297 |     Parameters | 
| 298 |     ---------- | 
| 299 |     x : | 
| 300 |         Array of nibbles. | 
| 301 |     is_leaf : | 
| 302 |         True if this is part of a leaf node, or false if it is an extension | 
| 303 |         node. | 
| 304 |  | 
| 305 |     Returns | 
| 306 |     ------- | 
| 307 |     compressed : `bytearray` | 
| 308 |         Compact byte array. | 
| 309 |  | 
| 310 |     """ | 
| 311 |     compact = bytearray() | 
| 312 | |
| 313 |     if len(x) % 2 == 0:  # ie even length | 
| 314 |         compact.append(16 * (2 * is_leaf)) | 
| 315 |         for i in range(0, len(x), 2): | 
| 316 |             compact.append(16 * x[i] + x[i + 1]) | 
| 317 |     else: | 
| 318 |         compact.append(16 * ((2 * is_leaf) + 1) + x[0]) | 
| 319 |         for i in range(1, len(x), 2): | 
| 320 |             compact.append(16 * x[i] + x[i + 1]) | 
| 321 | |
| 322 |     return Bytes(compact) | 
bytes_to_nibble_list
Converts a Bytes into to a sequence of nibbles (bytes with value < 16).
Parameters
bytes_:
The Bytes to convert.
Returns
nibble_list : Bytes
The Bytes in nibble-list format.
                def bytes_to_nibble_list(bytes_: Bytes) -> Bytes:
            
| 326 |     """ | 
|---|---|
| 327 |     Converts a `Bytes` into to a sequence of nibbles (bytes with value < 16). | 
| 328 |  | 
| 329 |     Parameters | 
| 330 |     ---------- | 
| 331 |     bytes_: | 
| 332 |         The `Bytes` to convert. | 
| 333 |  | 
| 334 |     Returns | 
| 335 |     ------- | 
| 336 |     nibble_list : `Bytes` | 
| 337 |         The `Bytes` in nibble-list format. | 
| 338 |  | 
| 339 |     """ | 
| 340 |     nibble_list = bytearray(2 * len(bytes_)) | 
| 341 |     for byte_index, byte in enumerate(bytes_): | 
| 342 |         nibble_list[byte_index * 2] = (byte & 0xF0) >> 4 | 
| 343 |         nibble_list[byte_index * 2 + 1] = byte & 0x0F | 
| 344 |     return Bytes(nibble_list) | 
_prepare_trie
Prepares the trie for root calculation. Removes values that are empty,
hashes the keys (if secured == True) and encodes all the nodes.
Parameters
trie :
The Trie to prepare.
get_storage_root :
Function to get the storage root of an account. Needed to encode
Account objects.
Returns
out : Mapping[ethereum.base_types.Bytes, Node]
Object with keys mapped to nibble-byte form.
                def _prepare_trie(trie: Trie[K, V], get_storage_root: Optional[Callable[[Address], Root]]) -> Mapping[Bytes, Bytes]:
            
| 351 |     """ | 
|---|---|
| 352 |     Prepares the trie for root calculation. Removes values that are empty, | 
| 353 |     hashes the keys (if `secured == True`) and encodes all the nodes. | 
| 354 |  | 
| 355 |     Parameters | 
| 356 |     ---------- | 
| 357 |     trie : | 
| 358 |         The `Trie` to prepare. | 
| 359 |     get_storage_root : | 
| 360 |         Function to get the storage root of an account. Needed to encode | 
| 361 |         `Account` objects. | 
| 362 |  | 
| 363 |     Returns | 
| 364 |     ------- | 
| 365 |     out : `Mapping[ethereum.base_types.Bytes, Node]` | 
| 366 |         Object with keys mapped to nibble-byte form. | 
| 367 |  | 
| 368 |     """ | 
| 369 |     mapped: MutableMapping[Bytes, Bytes] = {} | 
| 370 | |
| 371 |     for preimage, value in trie._data.items(): | 
| 372 |         if isinstance(value, Account): | 
| 373 |             assert get_storage_root is not None | 
| 374 |             address = Address(preimage) | 
| 375 |             encoded_value = encode_node(value, get_storage_root(address)) | 
| 376 |         else: | 
| 377 |             encoded_value = encode_node(value) | 
| 378 |         if encoded_value == b"": | 
| 379 |             raise AssertionError | 
| 380 |         key: Bytes | 
| 381 |         if trie.secured: | 
| 382 |             # "secure" tries hash keys once before construction | 
| 383 |             key = keccak256(preimage) | 
| 384 |         else: | 
| 385 |             key = preimage | 
| 386 |         mapped[bytes_to_nibble_list(key)] = encoded_value | 
| 387 | |
| 388 |     return mapped | 
root
Computes the root of a modified merkle patricia trie (MPT).
Parameters
trie :
Trie to get the root of.
get_storage_root :
Function to get the storage root of an account. Needed to encode
Account objects.
Returns
root : .fork_types.Root
MPT root of the underlying key-value pairs.
                def root(trie: Trie[K, V], get_storage_root: Optional[Callable[[Address], Root]]) -> Root:
            
| 395 |     """ | 
|---|---|
| 396 |     Computes the root of a modified merkle patricia trie (MPT). | 
| 397 |  | 
| 398 |     Parameters | 
| 399 |     ---------- | 
| 400 |     trie : | 
| 401 |         `Trie` to get the root of. | 
| 402 |     get_storage_root : | 
| 403 |         Function to get the storage root of an account. Needed to encode | 
| 404 |         `Account` objects. | 
| 405 |  | 
| 406 |  | 
| 407 |     Returns | 
| 408 |     ------- | 
| 409 |     root : `.fork_types.Root` | 
| 410 |         MPT root of the underlying key-value pairs. | 
| 411 |  | 
| 412 |     """ | 
| 413 |     obj = _prepare_trie(trie, get_storage_root) | 
| 414 | |
| 415 |     root_node = encode_internal_node(patricialize(obj, Uint(0))) | 
| 416 |     if len(rlp.encode(root_node)) < 32: | 
| 417 |         return keccak256(rlp.encode(root_node)) | 
| 418 |     else: | 
| 419 |         assert isinstance(root_node, Bytes) | 
| 420 |         return Root(root_node) | 
patricialize
Structural composition function.
Used to recursively patricialize and merkleize a dictionary. Includes memoization of the tree structure and hashes.
Parameters
obj : Underlying trie key-value pairs, with keys in nibble-list format. level : Current trie level.
Returns
node : ethereum.base_types.Bytes
Root node of obj.
                def patricialize(obj: Mapping[Bytes, Bytes], level: Uint) -> Optional[InternalNode]:
            
| 426 |     """ | 
|---|---|
| 427 |     Structural composition function. | 
| 428 |  | 
| 429 |     Used to recursively patricialize and merkleize a dictionary. Includes | 
| 430 |     memoization of the tree structure and hashes. | 
| 431 |  | 
| 432 |     Parameters | 
| 433 |     ---------- | 
| 434 |     obj : | 
| 435 |         Underlying trie key-value pairs, with keys in nibble-list format. | 
| 436 |     level : | 
| 437 |         Current trie level. | 
| 438 |  | 
| 439 |     Returns | 
| 440 |     ------- | 
| 441 |     node : `ethereum.base_types.Bytes` | 
| 442 |         Root node of `obj`. | 
| 443 |  | 
| 444 |     """ | 
| 445 |     if len(obj) == 0: | 
| 446 |         return None | 
| 447 | |
| 448 |     arbitrary_key = next(iter(obj)) | 
| 449 | |
| 450 |     # if leaf node | 
| 451 |     if len(obj) == 1: | 
| 452 |         leaf = LeafNode(arbitrary_key[level:], obj[arbitrary_key]) | 
| 453 |         return leaf | 
| 454 | |
| 455 |     # prepare for extension node check by finding max j such that all keys in | 
| 456 |     # obj have the same key[i:j] | 
| 457 |     substring = arbitrary_key[level:] | 
| 458 |     prefix_length = len(substring) | 
| 459 |     for key in obj: | 
| 460 |         prefix_length = min( | 
| 461 |             prefix_length, common_prefix_length(substring, key[level:]) | 
| 462 |         ) | 
| 463 |  | 
| 464 |         # finished searching, found another key at the current level | 
| 465 |         if prefix_length == 0: | 
| 466 |             break | 
| 467 | |
| 468 |     # if extension node | 
| 469 |     if prefix_length > 0: | 
| 470 |         prefix = arbitrary_key[int(level) : int(level) + prefix_length] | 
| 471 |         return ExtensionNode( | 
| 472 |             prefix, | 
| 473 |             encode_internal_node( | 
| 474 |                 patricialize(obj, level + Uint(prefix_length)) | 
| 475 |             ), | 
| 476 |         ) | 
| 477 | |
| 478 |     branches: List[MutableMapping[Bytes, Bytes]] = [] | 
| 479 |     for _ in range(16): | 
| 480 |         branches.append({}) | 
| 481 |     value = b"" | 
| 482 |     for key in obj: | 
| 483 |         if len(key) == level: | 
| 484 |             # shouldn't ever have an account or receipt in an internal node | 
| 485 |             if isinstance(obj[key], (Account, Receipt, Uint)): | 
| 486 |                 raise AssertionError | 
| 487 |             value = obj[key] | 
| 488 |         else: | 
| 489 |             branches[key[level]][key] = obj[key] | 
| 490 | |
| 491 |     subnodes = tuple( | 
| 492 |         encode_internal_node(patricialize(branches[k], level + Uint(1))) | 
| 493 |         for k in range(16) | 
| 494 |     ) | 
| 495 |     return BranchNode( | 
| 496 |         cast(BranchSubnodes, assert_type(subnodes, Tuple[Extended, ...])), | 
| 497 |         value, | 
| 498 |     ) |