Exploring the first 1000 epochs of eth2

We look at data between epochs 0 and 1000 (2020-12-05 22:40:23 UTC) and report updated metrics for eth2’s mainnet. You can also find a similar notebook for Medalla here and Pyrmont here.

Performance of duties

Anatomy of an aggregate

Before we begin, remember that the beacon chain is composed of blocks, each of which contains up to 128 aggregates. These aggregates are “vote summaries”, where each aggregate summarises the votes (attestations) of all members of a committee. Each epoch, validators are assigned to exactly one committee. An epoch is divided further into 32 slots, each slot being assigned several committees.

We’ll often provide examples in the following format. Each row is an aggregate attestation.

Let’s dissect it.

slot is the inclusion slot of the aggregate, i.e., the slot of the block which contains the aggregate. Note that an aggregate may be included multiple times (we’ll look at this below).
att_slot is the attestation slot of the aggregate. The aggregate summarises votes from one committee of a slot, called the attestation slot.
committee_index is the index of the committee the aggregate summarises votes from.
beacon_block is the root of the beacon block validators have voted for. Validators are expected to include in their attestation their best estimate of the current head of the chain following the GHOST fork choice rule. They are rewarded if the beacon_block in their vote corresponds to the head.
source_block is the root of the source checkpoint. To provide finality under the Friendly Finality Gadget (FFG) rules, validators include a reference to the source checkpoint, the last justified boundary block of an epoch (either the first block of an epoch or, if it is missing, the last block since).
target_block is the root of the target checkpoint. Effectively validators draw a link between the source and the target checkpoints, indicating that the target should be the next justified epoch.
attesting_indices is the voting summary. In the example above, we look at a committee of 3 validators (there are 3 digits in attesting_indices). 101 indicates that the aggregates contains a vote for the attributes above (attestation slot, committee index, beacon block, source and target blocks) from validators at indices 0 and 2 of the committee. Note that the validator at index 1 could have also published the same vote, but too late for aggregators to pick it up and include it in the summary.

Attester duties

We compare the number of included attestations with the number of expected attestations. Since each (active) validator is expected to attest exactly once per epoch, this also charts the percentage of validators attesting over time.

Proposer duties

How many blocks are there in the canonical chain? Validators are sampled randomly to produce blocks. If they fail to produce one, the block is missing. They also sometimes produce one but the block is orphaned, i.e., the chain continues on a different block. The plot below charts the proportion of included blocks to expected blocks.

Correctness of attestations

Target checkpoint

Attestations vouch for some target checkpoint to justify. We can check whether they vouched for the correct one by comparing their target_block with the latest known block root as of the start of the attestation epoch (that’s a mouthful). How many individual attestations (unique votes by validators) correctly attest for the target?

How does the correctness evolve over time?

Head of the chain

Attestations must also vote for the correct head of the chain, as returned by the GHOST fork choice rule. To check for correctness, one looks at the latest block known as of the attestation slot. Possibly, this block was proposed for the same slot as the attestation att_slot. When the beacon_block_root attribute of the attestation and the latest block root match, the head is correct!

How does the correctness evolve over time?

Inclusion delay

Attestations are rewarded for how fast they are included. An attestation must be included at least one slot later than its attestation slot and at most 32 slots later. The inclusion delay is taken to be the minimum inclusion time, and is thus contained between 1 and 32.

Aggregate attestations

eth2 is built to scale to hundreds of thousands of validators. This introduces overhead from message passing (and inclusion) when these validators are asked to vote on the canonical chain. To alleviate the beacon chain, votes (a.k.a. attestations) can be aggregated.

Since we expect validators to broadly agree in times of low latency, we also expect that a lot of attestations will vote for identical items. We can aggregate such a set of attestations \(I\) into a single aggregate.

When we have \(N\) active validators, about \(N / 32\) are selected to attest for each slot in an epoch. The validators for a slot \(s\) are further divided between a few committees. Identical votes from validators in the same committee can be aggregated. Assume that two aggregate attestations were formed from attestations of validators in set \(C(s, c)\), validators in committee \(c\) attesting for slot \(s\). One aggregate contains attestations from set \(I \subseteq C(s, c)\) and the other attestations from set \(J \subseteq C(s, c)\). We have two cases:

When the intersection of \(I\) and \(J\) is non-empty, we cannot aggregate the two aggregates further.
When the intersection of \(I\) and \(J\) is empty, the two aggregates can themselves be aggregated, into one containing attestations from validator set \(I \cup J\).

How many attestations are contained in aggregates?

To obtain this measure, we simply count how many 1s appear in the attesting_indices attribute of an aggregate.

We can plot the same, weighing by the size of the validator set in the aggregate, to count how many attestations each size of aggregates included.

Overall, we can plot the Lorenz curve of aggregate attestations. This allows us to find out the share of attestations held by the 20% largest aggregates.

The answer is 33%.

How much savings did aggregates provide?

In the previous plots, we “double-counted” some attestations which were included in several aggregates. Here, we tally the number of individual attestations, unique votes from validators. We compare how many individual attestations to how many aggregates were included in blocks.

We have 72.79 times more individual attestations than aggregates, meaning that if we were not aggregating, we would have 72.79 times as much data on-chain.

In how many aggregate attestations is an attestation included?

Individual attestations can be included in several aggregates. For each, we count how many aggregates it appears in, and produce the following histogram.

Most attestations were included in an aggregate once only.

How many (myopic) redundant aggregate attestations are there?

We call myopic redundant identical aggregate attestations (same five attributes and same set of validator indices) which are included in more than one block. It can happen when a block producer does not see that an aggregate was previously included (e.g., because of latency), or simply when the block producer doesn’t pay attention and greedily adds as many aggregates as they know about.

For instance, the two aggregates below are myopic redundant. They have the same attesting indices and same attributes, yet were published in two different blocks (notice the different slot).

We plot a histogram of the number of times an aggregate was included on-chain.

The mode is 1, which is also the optimal case. A redundant aggregate does not have much purpose apart from bloating the chain.

We could generalise the myopic redundant definition and call redundant an aggregate included in a block for which all of its attesting indices were previously seen in other aggregates.

In the example below, the third aggregate is redundant, though not myopic redundant. It is made up of indices all of which were published in previous blocks.

We find that 17564 aggregates are redundant out of 309426 included in blocks.

How many times did a block include the exact same aggregate attestation more than once?

We could call these strongly redundant, as this is pure waste. In the example below, the same block includes twice the same aggregate.

Currently, we never see identical aggregates included twice in the same block.

How many aggregates in a block are included by another aggregate in the same block?

We now define subset aggregates. Suppose two aggregates in the same block with equal attributes (attestation slot, committee index, beacon root, source root and target root) include validator sets \(I\) and \(J\) respectively. If we have \(I \subset J\), i.e., if all validators of the first aggregate are also included in the second aggregate (but the reverse is not true), then we call the first aggregate a subset aggregate of the second. In the example here, the second aggregate is a subset of the first.

Subset aggregates, like redundant aggregate attestations, can be removed from the finalised chain without losing any voting information. In fact, subset aggregates use much less local information than redundant aggregates. To root out subset aggregates, a client simply must ensure that no aggregate it is prepared to include in a block is a subset aggregate of another. Meanwhile, to root out redundant aggregates, a client must check all past blocks (until the inclusion limit of 32 slots) to ensure that it is not including a redundant aggregate. In a sense, subset aggregate are “worse” as they should be easier to root out.

So among all included aggregates in blocks, how many are subset aggregates? We count these instances for attestations included in blocks until epoch 1000 (2020-12-05 22:40:23 UTC).

We find that 0.27% included aggregates are subset aggregates.

How often are subset aggregates of size 1?

In Medalla, we observed that subset aggregates were often of size 1. In other words, frequently a “big” aggregate is included, aggregating very many validators, and then a second aggregate of size 1, namely, a simple attestation, is included too, while this simple attestation is already accounted for by the first, larger aggregate.

In Pyrmont, clients seemed to have improved their block-packing algorithms, since we did not find any subset aggregate of size 1.

Let’s see what Mainnet holds.

How many times were clashing attestations included in blocks?

We look at situations where two aggregate attestations are included in the same block, with identical attributes (same attesting slot, attesting committee, beacon chain head, source block and target block) but different attesting indices and neither one is a subset of the other. We define the following two notions, assuming the two aggregate attestations include attestations of validator sets \(I\) and \(J\) respectively:

Strongly clashing: The two aggregates share some validator indices, i.e., \(I \cap J \neq \emptyset\). The two aggregate attestations were incompatible, so could not be aggregated further. In the next example, the two aggregates are strongly clashing as the second attesting index is set to 1 for both, yet they each contain an attesting index the other does not have.

Weakly clashing: The two aggregates have different validator indices, i.e., \(I \cap J = \emptyset\). The two aggregate attestations could have been aggregated further. In the next example, the two aggregates are weakly clashing as they could have been aggregated further into attesting indices 111.

Let’s first count how many aggregates are strongly clashing in blocks before slot 32000.

How many are weakly clashing?

None! That’s pretty great. It means blocks always include the most aggregated possible attestations, and we have a local optimum to the aggregation problem. It also means aggregators are doing a great job at producing the most succinct attestations possible, while, as we have seen, lately above 99% of the expected attestations are seen on-chain.

Note that optimally aggregating a set of aggregates is NP-complete! Here is a reduction of the optimal aggregation problem to the graph colouring. Set aggregate attestations as vertices in a graph, with an edge drawn between two vertices if the validator sets of the two aggregates have a non-empty overlap. In the graph colouring, we look for the minimum number of colours necessary to assign a colour to each vertex such that two connected vertices do not have the same colour. All vertices who share the same colour have an empty overlap, and thus can be combined into an aggregate. The minimum number of colours necessary to colour the graph tells us how few aggregates were necessary to combine a given set of aggregates further.

Aggregates glossary

We’ve looked at aggregate attestations in a few different ways. We offer here a table to summarise the definitions we have introduced and associated statistics.

Name	Explanation	Statistics	Recommendation
Aggregate	Attestation summarising the vote of validators in a single committee	There are 309426 aggregates included from slot 0 to slot 32031	x
Individual attestation	A single, unique, validator vote	There are 22522856 individual attestations	x
Savings ratio	The ratio of individual attestations to aggregate attestations	The savings ratio is 72.79	Keep it up!
Redundant aggregate	An aggregate containing validator attestations which were all already included on-chain, possibly across several aggregates with different sets of attesting indices	There are 17564 redundant aggregates included, 5.68% of all aggregates	Don’t include these
Myopic redundant aggregate	An aggregate included more than once on-chain, always with the same attesting indices	There are 5027 myopic redundant aggregates, 28.62% of all redundant aggregates and 1.62% of all aggregates	These are redundant too: don’t include them either

In the next table, we present definitions classifying aggregates when two or more instances are included in the same block with the same five attributes (attesting slot and committee, beacon root, source root and target root).

Name	Explanation	Statistics	Recommendation
Strongly redundant aggregate	An aggregate included more than once in the same block	There are strongly redundant aggregates	Keep only one of the strongly redundant aggregates
Subset aggregate	If not strongly redundant, an aggregate fully contained in another aggregate included in the same block	There are 843 subset aggregates until slot 32000, 0.27% of all aggregates until slot 32000	Drop all subset aggregates
Strongly clashing aggregates	If not a subset aggregate, an aggregate with attesting indices \(I\) such that there exists another aggregate attesting for the same in the same block with attesting indices \(J\) and \(I \cap J \neq \emptyset\)	There are 35543 strongly clashing aggregates until slot 32000, 11.49% of all aggregates until slot 32000	These cannot be aggregated further. Do nothing
Weakly clashing aggregates	If not a strongly clashing aggregate, an aggregate with attesting indices \(I\) such that there exists another aggregate attesting for the same in the same block with attesting indices \(J\)	There are 0 weakly clashing aggregates until slot 32000, 0% of all aggregates until slot 32000	These can be aggregated further into one aggregate with attesting indices \(I \cup J\). In an ideal world, we have 0 weakly clashing aggregates

Size 1 aggregates are lone attestations… why weren’t they aggregated further?

Name	Explanation	Statistics	Recommendation
Subset aggregate of size 1	A subset aggregate which is an unaggregated individual attestation	There are 0 subset aggregates of size 1 until slot 32000, 0% of all subset aggregates until slot 32000	Definitely drop these
Aggregate of size 1	An attestation included without being aggregated	There are 65435 aggregates of size 1, 21.15% of all aggregates	Either it is weakly clashing, so aggregate it further; or it is a subset aggregate, so drop it; or it is a redundant, drop it; or it is new and never aggregated, keep it