Skip to content

Documentation

The execution-spec-tests documentation is generated via mkdocs and hosted remotely on Github Pages at ethereum.github.io/execution-spec-tests.

Prerequisites

uv sync --all-extras

Build the Documentation

One time build:

uv run mkdocs build

Do a pre-commit check: One time build and lint/type checking:

pip install tox-uv
tox -e docs

Local Deployment and Test

This runs continually: Deploys the site locally and re-generates the site upon modifications to docs/**/*.md or tests/**/*.py:

uv run mkdocs serve

For more help (including ensuring a clean build), see the gen_test_doc pytest plugin's documentation:

A pytest plugin that generates test case documentation for use in mkdocs.

It generates the top-level "Test Case Reference" section in EEST's mkdocs site.

Note:

  • No output directory is specified for the generated output; file IO occurs via the mkdocs-gen-files plugin. mkdocs serve writes intermediate files to our local docs/ directory and then copies it to the site directory. We modify docs/navigation.md and write all other output underneath docs/tests. If mkdocs is interrupted, these intermediate artifacts are left in docs/.

Usage:

Ensuring a clean build

In case mkdocs has polluted the docs/ directory with intermediate files, run:

git restore docs/navigation.md  # Careful if you have local modifications!
rm -rf docs/tests docs/docs site

To test doc generation, run the plugin without mkdocs:

uv run fill -p pytest_plugins.filler.gen_test_doc.gen_test_doc --gen-docs --fork=<fork> tests

Or to build and view the site:

uv run mkdocs serve

Remote Deployment and Versioning

The execution-specs-test docs are hosted on Github pages at the repo's Github pages. Versions are updated/deployed automatically as part of Github Actions, but this can also be performed on the command-line.

Our mkdocs configuration uses mike as a version provider. All deployments should be made via mike (whether as part of CI/CD or executed locally).

The deployed versions of the docs managed via mike are kept in the gh-pages branch. When you run mike it commits to this branch and optionally pushes the changes directly to remote.

Aliases

We currently have two aliases (which both point to main as of #998):

  • latest: the current state of the main branch.
  • development: the current state of the main branch.

These aliases point to specific versions, as configured below. It's possible to share links containing either of these aliases or to specific versions, i.e, the following are all valid links:

CI/CD: Doc Deployment via Github Actions

There are two workflows that automatically deploy updated/new versions of the docs:

Workflow yaml File What When
docs_main.yaml Update "main" version of docs Push to 'main' branch, (e.g., on PR merge)
docs_tags.yaml Deploy new version of docs; tag is used as version name Upon creating a tag matching v*

Build and Deployment (without alias update)

Build a new version and deploy it to remote (this version will then show up in the version selector list):

uv run mike deploy -rebase --push v1.2.3

Local deployment

If you deploy locally, the documentation will be built with any changes made in your local repository. Check out the tag to deploy tagged versions.

Build, Deploy and Update the Alias

Build, deploy and update the version an alias points to with:

uv run mike deploy --rebase --push --update-aliases v1.2.3 latest

where v1.2.3 indicates the version's name and development is the alias. This will overwrite the version if it already exists.

Updating the 'main' version locally

"main" is just a version name (intended to reflect that it is build from the main branch). However, mike will build the docs site from the current local repository state (including local modifications). Therefore, make sure you're on the HEAD of the main branch before executing (unless you know what you're doing 😉)!

uv run mike deploy --rebase --push main

If the alias accidentally go change:

uv run mike deploy --rebase --push --update-aliases main development

Viewing and Deleting Versions

List versions:

uv run mike list

Delete a version:

uv run mike delete v1.2.3a1-eof

Set Default Version

Set the default version of the docs to open upon loading the page:

uv run mike set-default --push latest

Typically, this must only be executed once for a repo.

Implementation

Plugins

The documentation flow uses mkdocs and the following additional plugins:

The "Test Case Reference" Section

This section is auto-generated via a combination of:

  1. mkdocstrings and mkdocstrings-python,
  2. mkdocs-gen-files,
  3. mkdocs-literate-nav.

It auto-generates a sequence of nested pages (with nav entries) of all python modules detected under ./tests. Each page contains a stub to the doc generated by mkdocstrings from the module's docstrings and source code. The mkdocs-gen-files and mkdocs-literate-nav plugins were created exactly for this purpose.

No action is necessary if a new test directory or module is added to ./tests, it will be picked up automatically.

Working with generated content

The files in the ./tests directory are watched by mkdocs serve. Run mkdocs serve and edit the source docstrings: The browser will reload with the new content automatically.

All pages that are to be included in the documentation and the navigation bar must be included in navigation.md, except "Test Case Reference" entries. This is enabled by mkdocs-literate-nav. The nav entries for the automatically generated "Test Case Reference" section are generated in mkdocs-gen-files and appended to navigation.md.

Current nav ordering limitations

The "Test Case Reference" section must currently be the last section in the nav. This is because our mkdocs flow:

  1. Reads navigation.md.
  2. Generates the Test Case Reference documentation and appends the Test Case Reference entries to navigation.md
  3. Generates the nav.

If necessary, we could probably split navigation.md into two files

  • navigation-pre-test-case-reference.md,
  • navigation-post-test-case-reference.md,

and create an arbitrary ordering in the Test Case Reference doc gen script. But this is untested.

Read the Docs

Originally, documentation was hosted at readthedocs.io. Currently, this now defunct page (execution-spec-tests.readthedocs.io) is configured to redirect to the Github Pages site. This is achieved by following the steps listed in the second half of this answer on stackoverflow. A public repo with a dummy Sphinx project is required to achieve this: danceratopz/est-docs-redirect.