Test Markers

Test markers are used to categorize tests and to run specific subsets of tests. They are defined in the test files using the pytest.mark decorator.

The examples below use StateTestFiller tests, but the same markers can also be applied to BlockchainTestFiller tests.

Fork Markers

These markers are used to specify the forks for which a test is valid.

@pytest.mark.valid_from("FORK_NAME")

Bases: ValidityMarker

Marker used to specify the fork from which the test is valid. The test will not be filled for forks before the specified fork.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.valid_from("London")
def test_something_only_valid_after_london(
    state_test: StateTestFiller,
    pre: Alloc
):
    pass

In this example, the test will only be filled for the London fork and after, e.g. London, Paris, Shanghai, Cancun, etc.

Source code in src/pytest_plugins/forks/forks.py

class ValidFrom(ValidityMarker):
    """
    Marker used to specify the fork from which the test is valid. The test will not be filled for
    forks before the specified fork.

    ```python
    import pytest

    from ethereum_test_tools import Alloc, StateTestFiller

    @pytest.mark.valid_from("London")
    def test_something_only_valid_after_london(
        state_test: StateTestFiller,
        pre: Alloc
    ):
        pass
    ```

    In this example, the test will only be filled for the London fork and after, e.g. London,
    Paris, Shanghai, Cancun, etc.
    """

    def _process_with_marker_args(self, *fork_args) -> Set[Fork]:
        """Process the fork arguments."""
        forks: Set[Fork] = self.process_fork_arguments(*fork_args)
        resulting_set: Set[Fork] = set()
        for fork in forks:
            resulting_set |= {f for f in self.all_forks if f >= fork}
        return resulting_set

@pytest.mark.valid_until("FORK_NAME")

Bases: ValidityMarker

Marker to specify the fork until which the test is valid. The test will not be filled for forks after the specified fork.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.valid_until("London")
def test_something_only_valid_until_london(
    state_test: StateTestFiller,
    pre: Alloc
):
    pass

In this example, the test will only be filled for the London fork and before, e.g. London, Berlin, Istanbul, etc.

Source code in src/pytest_plugins/forks/forks.py

class ValidUntil(ValidityMarker):
    """
    Marker to specify the fork until which the test is valid. The test will not be filled for
    forks after the specified fork.

    ```python
    import pytest

    from ethereum_test_tools import Alloc, StateTestFiller

    @pytest.mark.valid_until("London")
    def test_something_only_valid_until_london(
        state_test: StateTestFiller,
        pre: Alloc
    ):
        pass
    ```

    In this example, the test will only be filled for the London fork and before, e.g. London,
    Berlin, Istanbul, etc.
    """

    def _process_with_marker_args(self, *fork_args) -> Set[Fork]:
        """Process the fork arguments."""
        forks: Set[Fork] = self.process_fork_arguments(*fork_args)
        resulting_set: Set[Fork] = set()
        for fork in forks:
            resulting_set |= {f for f in self.all_forks if f <= fork}
        return resulting_set

@pytest.mark.valid_at_transition_to("FORK_NAME")

Bases: ValidityMarker

Marker to specify that a test is only meant to be filled at the transition to the specified fork.

The test usually starts at the fork prior to the specified fork at genesis and at block 5 (for pre-merge forks) or at timestamp 15,000 (for post-merge forks) the fork transition occurs.

import pytest

from ethereum_test_tools import Alloc, BlockchainTestFiller

@pytest.mark.valid_at_transition_to("London")
def test_something_that_happens_during_the_fork_transition_to_london(
    blockchain_test: BlockchainTestFiller,
    pre: Alloc
):
    pass

In this example, the test will only be filled for the fork that transitions to London at block number 5, BerlinToLondonAt5, and no other forks.

To see or add a new transition fork, see the ethereum_test_forks.forks.transition module.

Note that the test uses a BlockchainTestFiller fixture instead of a StateTestFiller, as the transition forks are used to test changes throughout the blockchain progression, and not just the state change of a single transaction.

This marker also accepts the following keyword arguments:

• subsequent_transitions: Force the test to also fill for subsequent fork transitions.
• until: Implies subsequent_transitions and puts a limit on which transition fork will the test filling will be limited to.

For example:

@pytest.mark.valid_at_transition_to("Cancun", subsequent_transitions=True)

produces tests on ShanghaiToCancunAtTime15k and CancunToPragueAtTime15k, and any transition fork after that.

And:

@pytest.mark.valid_at_transition_to("Cancun", subsequent_transitions=True, until="Prague")

produces tests on ShanghaiToCancunAtTime15k and CancunToPragueAtTime15k, but no forks after Prague.

Source code in src/pytest_plugins/forks/forks.py

class ValidAtTransitionTo(ValidityMarker, mutually_exclusive=True):
    """
    Marker to specify that a test is only meant to be filled at the transition to the specified
    fork.

    The test usually starts at the fork prior to the specified fork at genesis and at block 5 (for
    pre-merge forks) or at timestamp 15,000 (for post-merge forks) the fork transition occurs.

    ```python
    import pytest

    from ethereum_test_tools import Alloc, BlockchainTestFiller

    @pytest.mark.valid_at_transition_to("London")
    def test_something_that_happens_during_the_fork_transition_to_london(
        blockchain_test: BlockchainTestFiller,
        pre: Alloc
    ):
        pass
    ```

    In this example, the test will only be filled for the fork that transitions to London at block
    number 5, `BerlinToLondonAt5`, and no other forks.

    To see or add a new transition fork, see the `ethereum_test_forks.forks.transition` module.

    Note that the test uses a `BlockchainTestFiller` fixture instead of a `StateTestFiller`,
    as the transition forks are used to test changes throughout the blockchain progression, and
    not just the state change of a single transaction.

    This marker also accepts the following keyword arguments:

    - `subsequent_transitions`: Force the test to also fill for subsequent fork transitions.
    - `until`: Implies `subsequent_transitions` and puts a limit on which transition fork will the
        test filling will be limited to.

    For example:
    ```python
    @pytest.mark.valid_at_transition_to("Cancun", subsequent_transitions=True)
    ```

    produces tests on `ShanghaiToCancunAtTime15k` and `CancunToPragueAtTime15k`, and any transition
    fork after that.

    And:
    ```python
    @pytest.mark.valid_at_transition_to("Cancun", subsequent_transitions=True, until="Prague")
    ```

    produces tests on `ShanghaiToCancunAtTime15k` and `CancunToPragueAtTime15k`, but no forks after
    Prague.
    """

    def _process_with_marker_args(
        self, *fork_args, subsequent_forks: bool = False, until: str | None = None
    ) -> Set[Fork]:
        """Process the fork arguments."""
        forks: Set[Fork] = self.process_fork_arguments(*fork_args)
        until_forks: Set[Fork] | None = (
            None if until is None else self.process_fork_arguments(until)
        )
        if len(forks) == 0:
            pytest.fail(
                f"'{self.test_name}': Missing fork argument with 'valid_at_transition_to' "
                "marker."
            )

        if len(forks) > 1:
            pytest.fail(
                f"'{self.test_name}': Too many forks specified to 'valid_at_transition_to' "
                "marker."
            )

        resulting_set: Set[Fork] = set()
        for fork in forks:
            resulting_set |= transition_fork_to(fork)
            if subsequent_forks:
                for transition_forks in (
                    transition_fork_to(f) for f in self.all_forks if f > fork
                ):
                    for transition_fork in transition_forks:
                        if transition_fork and (
                            until_forks is None
                            or any(transition_fork <= until_fork for until_fork in until_forks)
                        ):
                            resulting_set.add(transition_fork)
        return resulting_set

Fork Covariant Markers

These markers are used in conjunction with the fork validity markers to automatically parameterize tests with values that are valid for the fork being tested.

@pytest.mark.with_all_tx_types

This marker is used to automatically parameterize a test with all transaction types that are valid for the fork being tested.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.with_all_tx_types
@pytest.mark.valid_from("Berlin")
def test_something_with_all_tx_types(
    state_test: StateTestFiller, 
    pre: Alloc,
    tx_type: int
):
    pass

In this example, the test will be parameterized for parameter tx_type with values [0, 1] for fork Berlin, but with values [0, 1, 2] for fork London (because of EIP-1559).

@pytest.mark.with_all_contract_creating_tx_types

This marker is used to automatically parameterize a test with all contract creating transaction types that are valid for the fork being tested.

This marker only differs from pytest.mark.with_all_tx_types in that it does not include transaction type 3 (Blob Transaction type) on fork Cancun and after.

@pytest.mark.with_all_precompiles

This marker is used to automatically parameterize a test with all precompiles that are valid for the fork being tested.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.with_all_precompiles
@pytest.mark.valid_from("Shanghai")
def test_something_with_all_precompiles(
    state_test: StateTestFiller, 
    pre: Alloc,
    precompile: int,
):
    pass

In this example, the test will be parameterized for parameter precompile with values [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] for fork Shanghai, but with values [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] for fork Cancun which introduced the point evaluation precompile defined in EIP-4844.

@pytest.mark.with_all_evm_code_types

This marker is used to automatically parameterize a test with all EVM code types that are valid for the fork being tested.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.with_all_evm_code_types
@pytest.mark.valid_from("Frontier")
def test_something_with_all_evm_code_types(
    state_test: StateTestFiller,     
    pre: Alloc,
):
    pass

In this example, the test will be parameterized for parameter evm_code_type only with value [EVMCodeType.LEGACY] starting on fork Frontier, and eventually it will be parametrized with with values [EVMCodeType.LEGACY, EVMCodeType.EOF_V1] on the EOF activation fork.

In all calls to pre.deploy_contract, if the code parameter is Bytecode type, and evm_code_type==EVMCodeType.EOF_V1, the bytecode will be automatically wrapped in an EOF V1 container.

Code wrapping might fail in the following circumstances:

• The code contains invalid EOF V1 opcodes.
• The code does not end with a valid EOF V1 terminating opcode (such as Op.STOP or Op.REVERT or Op.RETURN).

In the case where the code wrapping fails, evm_code_type can be added as a parameter to the test and the bytecode can be dynamically modified to be compatible with the EOF V1 container.

One thing to note is that evm_code_type is not necessary to be added as a parameter to the test because the pre: Alloc fixture automatically consumes this fixture, and therefore it only needs to be added to the test signature if the test's logic needs it.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller
from ethereum_test_vm import EVMCodeType
from ethereum_test_vm import Opcodes as Op

@pytest.mark.with_all_evm_code_types
@pytest.mark.valid_from("Frontier")
def test_something_with_all_evm_code_types(
    state_test: StateTestFiller,
    pre: Alloc,
    evm_code_type: EVMCodeType
):
    code = Op.SSTORE(1, 1)
    if evm_code_type == EVMCodeType.EOF_V1:
        # Modify the bytecode to be compatible with EOF V1 container
        code += Op.STOP
    pre.deploy_contract(code)
    ...

@pytest.mark.with_all_call_opcodes

This marker is used to automatically parameterize a test with all EVM call opcodes that are valid for the fork being tested.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller
from ethereum_test_vm import Opcodes as Op

@pytest.mark.with_all_call_opcodes
@pytest.mark.valid_from("Frontier")
def test_something_with_all_call_opcodes(
    state_test: StateTestFiller,
    pre: Alloc,
    call_opcode: Op
):
    pass

In this example, the test will be parametrized for parameter call_opcode with values [Op.CALL, Op.CALLCODE] starting on fork Frontier, [Op.CALL, Op.CALLCODE, Op.DELEGATECALL] on fork Homestead, [Op.CALL, Op.CALLCODE, Op.DELEGATECALL, Op.STATICCALL] on fork Byzantium, and eventually it will be parametrized with with values [Op.CALL, Op.CALLCODE, Op.DELEGATECALL, Op.STATICCALL, Op.EXTCALL, Op.EXTSTATICCALL, Op.EXTDELEGATECALL] on the EOF activation fork.

Parameter evm_code_type will also be parametrized with the correct EVM code type for the opcode under test.

@pytest.mark.with_all_create_opcodes

This marker is used to automatically parameterize a test with all EVM create opcodes that are valid for the fork being tested.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller
from ethereum_test_vm import Opcodes as Op

@pytest.mark.with_all_create_opcodes
@pytest.mark.valid_from("Frontier")
def test_something_with_all_create_opcodes(
    state_test: StateTestFiller,
    pre: Alloc,
    create_opcode: Op
):
    pass

In this example, the test will be parametrized for parameter create_opcode with values [Op.CREATE] starting on fork Frontier, [Op.CREATE, Op.CREATE2] starting on fork Constantinople, and eventually it will be parametrized with with values [Op.CREATE, Op.CREATE2, Op.EOFCREATE] on the EOF activation fork.

Parameter evm_code_type will also be parametrized with the correct EVM code type for the opcode under test.

@pytest.mark.with_all_system_contracts

This marker is used to automatically parameterize a test with all system contracts that are valid for the fork being tested.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller
from ethereum_test_base_types import Address

@pytest.mark.with_all_system_contracts
@pytest.mark.valid_from("Cancun")
def test_something_with_all_system_contracts(
    state_test: StateTestFiller,
    pre: Alloc,
    system_contract: Address,
):
    pass

In this example, the test will be parameterized for parameter system_contract with value [0x000F3DF6D732807EF1319FB7B8BB8522D0BEAC02] for fork Cancun.

Covariant Marker Keyword Arguments

All fork covariant markers accept the following keyword arguments:

selector

A lambda function that can be used to filter the fork covariant values that are valid for this specific test.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.with_all_tx_types(selector=lambda tx_type: tx_type != 2)
@pytest.mark.valid_from("London")
def test_something_with_all_tx_types(
    state_test: StateTestFiller, 
    pre: Alloc,
    tx_type: int
):
    pass

Ideally, the lambda function should be used to explicitly filter out values that are not compatible with the test (exclusive filter), rather than explicitly selecting values (inclusive filter), as the parametrized values might change with future forks.

marks

A marker, list of markers, or a lambda function that can be used to add additional markers to the test.

import pytest

@pytest.mark.with_all_tx_types(
    marks=lambda tx_type: pytest.mark.skip("incompatible") if tx_type == 1 else None,
)
@pytest.mark.valid_from("London")
def test_something_with_all_tx_types_but_skip_type_1(state_test_only, tx_type):
    assert tx_type != 1
    ...

In this example, the test will be skipped if tx_type is equal to 1 by returning a pytest.mark.skip marker, and return None otherwise.

@pytest.mark.parametrize_by_fork

A test can be dynamically parametrized based on the fork using the parametrize_by_fork marker.

This marker takes two positional arguments:

• argnames: A list of parameter names that will be parametrized using the custom function.
• fn: A function that takes the fork as parameter and returns a list of values that will be used to parametrize the test at that specific fork.

And one keyword argument:

• marks (optional): A marker, list of markers, or a lambda function that can be used to add additional markers to the generated tests.

The marked test function will be parametrized by the values returned by the fn function for each fork.

If the parameters that are being parametrized is only a single parameter, the return value of fn should be a list of values for that parameter.

If the parameters that are being parametrized are multiple, the return value of fn should be a list of tuples/lists, where each tuple contains the values for each parameter.

import pytest

def covariant_function(fork):
    return [[1, 2], [3, 4]] if fork.name() == "Paris" else [[4, 5], [5, 6], [6, 7]]

@pytest.mark.parametrize_by_fork("test_parameter,test_parameter_2", covariant_function)
@pytest.mark.valid_from("Paris")
@pytest.mark.valid_until("Shanghai")
def test_case(state_test_only, test_parameter, test_parameter_2):
    pass

In this example, the test will be parametrized with the values [1, 2] and [3, 4] for the Paris fork, with values 1 and 3 being assigned to test_parameter and 2 and 4 being assigned to test_parameter_2. For the Shanghai fork, the test will be parametrized with the values [4, 5], [5, 6], and [6, 7]. Therefore, more test cases will be generated for the Shanghai fork.

If the parameters that are being parametrized is only a single parameter, the return value of fn should be a list of values for that parameter.

If the parameters that are being parametrized are multiple, the return value of fn should be a list of tuples/lists, where each tuple contains the values for each parameter.

The function can also return a list of pytest.param objects, which allows for additional markers and test IDs to be added to the test.

Fill/Execute Markers

These markers are used to apply different markers to a test depending on whether it is being filled or executed.

@pytest.mark.fill

This marker is used to apply markers to a test when it is being filled.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.fill(pytest.mark.skip(reason="Only for execution"))
def test_something(
    state_test: StateTestFiller, 
    pre: Alloc
):
    pass

In this example, the test will be skipped when it is being filled.

@pytest.mark.execute

This marker is used to apply markers to a test when it is being executed.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.execute(pytest.mark.xfail(reason="Depends on block context"))
def test_something(
    state_test: StateTestFiller, 
    pre: Alloc
):
    pass

In this example, the test will be marked as expected to fail when it is being executed, which is particularly useful so that the test is still executed but does not fail the test run.

Other Markers

@pytest.mark.slow

This marker is used to mark tests that are slow to run. These tests are not run during tox checks, and are only run when a release is being prepared.

@pytest.mark.pre_alloc_modify

This marker is used to mark tests that modify the pre-alloc in a way that would be impractical to reproduce in a real-world scenario.

Examples of this include:

• Modifying the pre-alloc to have a balance of 2^256 - 1.
• Address collisions that would require hash collisions.

@pytest.mark.skip()

This marker can be used to skip a test.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.skip(reason="Not implemented")
def test_something(state_test: StateTestFiller, pre: Alloc):
    pass

@pytest.mark.xfail()

This marker can be used to mark a test as expected to fail.

import pytest

from ethereum_test_tools import Alloc, StateTestFiller

@pytest.mark.xfail(reason="EVM binary doesn't support this opcode")
def test_something(state_test: StateTestFiller, pre: Alloc):
    pass