Documentation

The execution-spec-tests documentation is generated via mkdocs and hosted remotely on Github Pages at ethereum.github.io/execution-spec-tests.

Prerequisites

pip install -e .[docs]

Build the Documentation

One time build:

mkdocs build

Local Deployment and Test

This runs continually and re-generates the documentation upon changes in the ./docs/ sub-director and deploys the site locally (127.0.0.1:8000, by default):

mkdocs serve

Note: The gen-files plugin currently breaks the serve command by continually re-generating the documentation. Disable this config in mkdocs.yml to avoid this behaviour.

Test Remote Deployment

This can be used to generate and deploy a local version of the documentation remotely on Github pages in order to share a preview with other developers. Note, as the documentation is generated locally, even changes that have not been pushed will be deployed:

mkdocs gh-deploy

It will be deployed to the Github pages of the repo's username (branch is ignored), e.g., https://danceratopz.github.io/execution-spec-tests.

Implementation

Plugins

The documentation flow uses mkdocs and the following additional plugins:

• mkdocs: The main doc generation tool.
• mkdocs-material: Provides many additional features and styling for mkdocs.
• mkdocstrings and mkdocstrings-python: To generate documentation from Python docstrings.
• mkdocs-gen-files: To generate markdown files automatically for each test case Python module. See this page for example usage. This plugin is used to programmatically generate the nav section for the generated test case reference documentation.
• mkdocs-literate-nav: Is used to define the navigation layout for non-generated content and was created to work well with mkdocs-gen-files to add nav content for generated content.
• mkdocs-git-authors-plugin: To display doc contributors in the page footer.
• mkdocs-glightbox - for improved image and inline content display.

The "Test Case Reference" Section

This section is auto-generated via a combination of:

1. mkdocstrings and mkdocstrings-python,
2. mkdocs-gen-files,
3. mkdocs-literate-nav.

It auto-generates a sequence of nested pages (with nav entries) of all python modules detected under ./tests. Each page contains a stub to the doc generated by mkdocstrings from the module's docstrings and source code. The mkdocs-gen-files and mkdocs-literate-nav plugins were created exactly for this purpose.

No action is necessary if a new test directory or module is added to ./tests, it will be picked up automatically.

Working with generated content

The files in the ./tests directory are watched by mkdocs serve. Run mkdocs serve and edit the source docstrings: The browser will reload with the new content automatically.

Navigation

All pages that are to be included in the documentation and the navigation bar must be included in navigation.md, except "Test Case Reference" entries. This is enabled by mkdocs-literate-nav. The nav entries for the automatically generated "Test Case Reference" section are generated in mkdocs-gen-files and appended to navigation.md.

Current nav ordering limitations

The "Test Case Reference" section must currently be the last section in the nav. This is because our mkdocs flow:

1. Reads navigation.md.
2. Generates the Test Case Reference documentation and appends the Test Case Reference entries to navigation.md
3. Generates the nav.

If necessary, we could probably split navigation.md into two files

• navigation-pre-test-case-reference.md,
• navigation-post-test-case-reference.md,

and create an arbitrary ordering in the Test Case Reference doc gen script. But this is untested.