Introduction

This book is about Trin, which is software used to interact with the Ethereum protocol via the Portal Network.

Trin is a Portal network client which acts as a json-rpc server with:

• Nearly instant sync
• Low CPU & storage usage

The Ethereum protocol will allow full nodes to forget old data in an likely future upgrade. Portal network nodes can supply users with that data.

Trin makes it possible to access Ethereum with less computer resources than a regular full node. It does this by spreading data amongst peers.

flowchart TB
    subgraph Full node data: on one computer
    full[Regular full node]
    end
    subgraph Full node data: spread amongst computers
    p1[Portal node]
    p2[Portal node]
    p3[Portal node]
    p1 <--> p2
    p2 <--> p3
    p1 <--> p3

    end

🏗 The sections, content and links of this book are subject to change.

Portal Network

The portal network is a response to two needs. Users should have the ability to:

• Access Ethereum using peers (not providers) from 'small' computers.
  • An old nice-to-have.
• Access historical data once "history expiry" upgrade goes live
  • A likely future need.

What is "history expiry"

EIP-4444: Bound Historical Data in Execution Clients is an upgrade that seeks to limit the costs of participating in the network. It does this by allowing the clearing of data older than 1 year.

How the Portal network works

Small users working together

graph TD;
    A & B & C & D & E & F & G & H & I --> id5[complete network data];

The portal network consists of many small nodes that each contribute to the whole. Each node is allocated a specific part of the network to obtain from peers and serve back to the network.

The portal network splits data in to different types (e.g., blocks vs new transactions). Each distinct type is effectively a new network. A portal client such as Trin can be used to operate on each/all of these different sub-protocols.

Dedicated sub-protocols

Users can elect to be part of some sub-networks:

graph TD;
    A & B & C --> id1[(History)]
    D & E & F --> id2[(State)]
    A & G & I --> id4[(Indices)]
    C & E & H --> id3[(Txs)]
    id1[(History)] & id2[(State)] & id3[(Txs)] & id4[(Indices)] --> id5[complete network data];

Peer referrals based on names

Nodes make requests to each other for data. If they don't have the data, they look at their peers and suggest one that is most likely to.

graph LR;
    id1[A requests data] -.-> D & E
    D -.-> F & G
    id1[A requests data] ---> B --->  id2[C has data]
    E -.-> F
    B & G -.-> I

Standard requests

Each node has a name, and only holds data that is similar to that name. Peers can tell who is likely to have what data based on these names.

sequenceDiagram
    Alice-->>Bob: Looking for data 0xabc
    Bob-->>Alice: Sorry, but try Charlie (gives address)
    Alice-->>Charlie: Looking for data 0xabc
    Charlie-->>Alice: I have it, do you want?
    Alice-->>Charlie: Yes
    Charlie->>Alice: Data 0xabc

Tunable resources

Nodes keep content that is similar to their name. That similarity radius can be made larger to voluntarily hold more data.

graph TD;
    id1[(Alice with big hard drive)]
    id2[(Bob)]
    id4[(Charlie, medium)]

In addition to Trin, other portal clients are in development and participate in the same network.

Users

The following are users who are well-suited to using Trin.

Laptop wallet user

A user has a laptop that frequently is turned off. When they want to transact, they can turn on Trin and connect their wallet to it.

Benefit: Wallet use without reliance on third party wallet APIs.

Desktop wallet user

A user has a desktop that usually on, but most of the disk is used for other things. When they want to transact, their wallet is already connected to their portal node.

Benefit: Wallet use without reliance on third party wallet APIs. Contributes to network health without using entire disk.

Protocol experimentation

A researcher looking to explore the Ethereum protocol, testing out specific aspects and perhaps making experimental changes to the protocol.

Benefit: Spin up a node and play around quickly and with low cost.

Single board computer hobbyist

A raspberry pi 3, or similarly-sized computer with could contribute to network health.

Currently a raspberry pi 4 can run a full node, with consensus and execution clients, however this is a bit tight and requires a ~2TB SSD.

Benefit: Learn about Ethereum, get node access and provide the network with additional robustness.

Mobile user

Trin is not currently configured to run on mobile, however this is plausibly a viable and interesting use case. A trin node could run as a background task with configurable limits on disk, CPU and bandwidth use.

Benefit: Wallet use without reliance on third party wallet APIs. Contributes to network health.

Unsuitable users

There are situations where Trin is estimated to not be a good node choice:

• Time-critical chain tip data. Likely that data distribution may not be fast enough for these use cases, however testing may show otherwise.
  • Consensus participation. Beacon chain staking with a Consensus client with Portal Network node as Execution client.
  • Block builder. Serving blocks to beacon chain validator nodes via MEV-boost
• Data analysis requiring state at historical blocks. Trin is not an archive node and does not expose trace_* or debug_* endpoints.

Requirements

Hardware

Suitable:

• Processor: x86 or Arm based. Minimum spec TBD.
• RAM: Minimum TBD.
• Disk: Any.

Testing and reports of performance on the following are welcome:

• RISC-V based processor.
• Resource constrained (CPU/RAM)

Software

• Unix based operating system
• Rust installation (minimum v1.66)

Network

Testing/reports of low-bandwidth network are welcome.

Trin should be compatible with VPN use, but if you experience difficulty connecting to the network we recommend disabling your VPN.

Startup

Configuration occurs at startup via flags:

cargo run -p trin -- --flag1 value --flag2 value

Backslashes allow flags to be on new lines for easier reading.

cargo run -p trin -- \
    --flag1 value \
    --flag2 value \
    --flag3 value1,value2 \

Flags

For the most up to date flags run:

cargo run -p trin -- --help

Bootnodes

Trin automatically connects to the Portal Network bootnodes. Use the --bootnodes cli flag to connect to a specific node or to none.

Control disk use

Trin can be tuned to control how much disk space is used:

See the --mb flag.

Private Key management

Trin requires a private key to configure a node's identity. Upon startup, Trin will automatically generate a random private key that will be re-used every time Trin is restarted. The only exceptions are if...

• User supplies a private key via the --unsafe-private-key flag, in which case that private key will be used to create the node's identity.
• User deletes the TRIN_DATA_DIR or changes the TRIN_DATA_DIR. In which case a new private key will be randomly generated and used.

Sub-Protocols

Trin can connect to different sub-protocols to have access to different types of data. One more more can be selected, but be aware that not all sub-protocols are ready:

• Execution State Network
• Execution History Network
• Execution Transaction Gossip Network
• Execution Canonical Indices Network

Networking configuration

Optionally one can specify Trin's network properties:

• What sort of network connections (HTTP vs IPC)
• Port answering Ethereum-related queries
• Port for connecting to other nodes

These types of flags have defaults.

Use

Once Trin is running, it will be serving Ethereum data in response to requests. This can be accessed by other programs, such as a wallet in a web browser.

Once Trin is running, another program will be able to communicate with Trin as it would any other Ethereum node.

Additionally, commands can be made in the terminal to test functionality. See sections below for more detail.

Access from different computer

If Trin is started on host computer by user, serving data over HTTP port then the following command can be issued on another computer to send requests to Trin and receive responses:

ssh -N -L <port>:127.0.0.1:<port> <user>@<host>

For example:

ssh -N -L 8545:127.0.0.1:8545 username@mycomputer

Accessing Trin from another computer using IPC is not covered here.

Making queries

If you want to manually query trin, the following patterns can be used, depending on whether Trin was started with --web3-transport as http or ipc.

Query form

A query for JSON-RPC has the following form for a call to "methodname" that accepts two parameters: parameter_one and parameter_two.

Query:

{
    "jsonrpc": "2.0",
    "method": "<methodname>",
    "params": ["<parameter_one>", "<parameter_two>"],
    "id":1
}

Usually passed on one line:

{"jsonrpc":"2.0","method":"<methodname>","params":["<parameter_one>", "<parameter_two>"],"id":1}

HTTP transport

Command for query (above) to HTTP server on port:

curl -X POST -H "Content-Type: application/json" -d '<query>' localhost:<port> | jq

IPC transport

Command for query (above) to IPC server with socket file located at /path/to/ipc:

echo '<query>' | nc -U </path/to/ipc> | jq

Response

If the data is not in the network the following response is expected:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": "0x"
}

Ethereum data

Trin is designed to eventually serve the JSON-RPC methods that an Ethereum full node would provide. This includes methods the start with the eth_ namespace.

Here is an example of making an eth_blockNumber request to a node serving over HTTP to get the latest block number.

{"jsonrpc": "2.0", "method": "eth_blockNumber", "params": [], "id":1}

HTTP

curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "method": "eth_blockNumber", "params": [], "id":1}' localhost:8545 | jq

IPC

echo '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' | nc -U /tmp/trin-jsonrpc.ipc | jq

Portal network data

There are methods for requesting data that are specific to:

• Each sub-protocol (history, state, etc.)
  • portal_history*
  • portal_state*
• Discovery protocol
  • discv5_*

See the Portal Network JSON-RPC specification here for a comprehensive and interactive view of specific methods available.

Designing a Query

One can identify data by its "content key". The following queries ask Trin to speak with peers, looking for a particular piece of data.

Let us request the block body for block 16624561

• Block hash: 0xd27f5e55d88b447788667b3d72cca66b7c944160f68f0a62aaf02aa7e4b2af17
• Selector for a block body: 0x01 (defined in Portal Network spec under the History sub-protocol).
• Content key: 0x01d27f5e55d88b447788667b3d72cca66b7c944160f68f0a62aaf02aa7e4b2af17
• Request: portal_historyRecursiveFindContent, which accepts a content key as a parameter

{"jsonrpc":"2.0","method":"portal_historyRecursiveFindContent","params":["0x01d27f5e55d88b447788667b3d72cca66b7c944160f68f0a62aaf02aa7e4b2af17"],"id":1}

HTTP

curl -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"portal_historyRecursiveFindContent","params":["0x01d27f5e55d88b447788667b3d72cca66b7c944160f68f0a62aaf02aa7e4b2af17"],"id":1}' http://localhost:8545 | jq

IPC

echo '{"jsonrpc":"2.0","method":"portal_historyRecursiveFindContent","params":["0x01d27f5e55d88b447788667b3d72cca66b7c944160f68f0a62aaf02aa7e4b2af17"],"id":1}' | nc -U /tmp/trin-jsonrpc.ipc | jq

Monitoring

Once Trin is running, the following may be useful

Logs

If errors are encountered, they will be logged to the console in which Trin was started.

Be aware that The RUST_LOG variable allows for control of what logs are visible.

• RUST_LOG=info cargo run -p trin
• RUST_LOG=debug cargo run -p trin

If started as a systemd service logs will be visible with:

journalctl -fu <trin-service-name>.service

Disk use

The following locations are where trin stores data by default:

• Mac Os: ~/Library/Application Support/trin
• Unix-like: $HOME/.local/share/trin

cd /path/to/data
du -sh

CPU and memory use

htop can be used to see the CPU and memory used by trin

• Ubuntu: sudo apt install htop
• Mac Os: brew install htop

htop

Metrics

Metrics setup no docker

Metrics setup with docker

Metrics setup no docker

Prometheus maintains a database of metrics (trin & system). Grafana converts metrics into graphs. Node exporter provides system information.

graph TD;
Browser-->Grafana-->Prometheus
Prometheus-->Trin & node[Node Exporter]

Install prometheus

Download the latest prometheus https://prometheus.io/download/

curl -LO <link>

Checksum

sha256sum <filename>

Extract

tar xvf <filename>

The directory will contain the binaries for prometheus and promtool. Copy these.

cd <dirname>
sudo cp prometheus /usr/local/bin/
sudo cp promtool /usr/local/bin/

Copy the console files

sudo cp -r consoles /etc/prometheus
sudo cp -r console_libraries /etc/prometheus

Remove the downloaded files

cd ~
rm <tar filename>
rm -r <extracted prometheus directory>

Make a prometheus user

sudo useradd --no-create-home --shell /bin/false prometheus

Create a prometheus data directory

sudo mkdir -p /var/lib/prometheus

Make a config file

sudo nano /etc/prometheus/prometheus.yml

Put this in the config file:

global:
  scrape_interval:     15s
  evaluation_interval: 15s

alerting:
  alertmanagers:
  - static_configs:
    - targets:
rule_files:

scrape_configs:
  - job_name: 'node_exporter'
    static_configs:
      - targets: ['localhost:9100']
  - job_name: 'trin'
    static_configs:
      - targets: ['localhost:9101']

The node_exporter job will gather system data by listening to port 9100. The trin job will gather system data by listening to port 9101.

Update the permissions

sudo chown -R prometheus:prometheus /etc/prometheus
sudo chown -R prometheus:prometheus /var/lib/prometheus

Prometheus will use port 9090 by default. Check it is not used by something else:

sudo lsof -i:9090

Create a service for prometheus

sudo nano /etc/systemd/system/prometheus.service

Include the following, pick another port if 9090 is already in use.

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=prometheus
Group=prometheus
Restart=always
RestartSec=5
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries \
    --web.listen-address="localhost:9090"
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

Start the service

sudo systemctl daemon-reload
sudo systemctl start prometheus
sudo systemctl status prometheus
sudo systemctl enable prometheus

Install node exporter

Download the latest node exporter https://prometheus.io/download/#node_exporter

curl -LO <link>

Checksum

sha256sum <filename>

Extract

tar xvf <filename>

The directory will contain the binary for node exporter. Copy this.

cd <dirname>
sudo cp node_exporter /usr/local/bin/

Remove the downloaded files

cd ~
rm <tar filename>
rm -r <extracted node_exporter directory>

Make a node_exporter user and give it permission to the binary.

sudo useradd --no-create-home --shell /bin/false node_exporter
sudo chown -R node_exporter:node_exporter /usr/local/bin/node_exporter

Make a service file:

sudo nano /etc/systemd/system/node_exporter.service

Start the service

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl status node_exporter
sudo systemctl enable node_exporter

Node explorer uses port 9100 by default.

Install grafana

Install

sudo apt-get install -y apt-transport-https software-properties-common wget
sudo wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
sudo apt update
sudo apt install grafana

Open config

sudo nano /etc/grafana/grafana.ini

Modify the http_adr line to use localhost

[server]
;http_addr = # Before
http_addr = localhost # After

Start grafana

sudo systemctl daemon-reload
sudo systemctl start grafana-server
sudo systemctl status grafana-server
sudo systemctl enable grafana-server

This will serve metrics over port 3000.

Generate a grafana dashboard. From trin root directory:

cargo run -p trin -- create-dashboard http://localhost:3000 admin admin http://127.0.0.1:9090

This will create a new monitoring database for trin. This will be visible in the grafana GUI, or directly at a URL similar to: http://localhost:3000/d/trin-app-metrics/trin-app-metrics

If you would like to run the create-dashboard command again, the data source and the dashboard must be deleted, which can be done in the grafana GUI.

Start trin with metrics on

The metrics port must match the trin job set in: /etc/prometheus/prometheus.yml.

cargo run -p trin -- \
    --enable-metrics-with-url 127.0.0.1:<metrics job port> \
    --web3-http-address http://127.0.0.1:<http port> \
    --web3-transport http

For example:

cargo run -p trin -- \
    --enable-metrics-with-url 127.0.0.1:9101 \
    --web3-http-address http://127.0.0.1:8545 \
    --web3-transport http

Updating metrics dashboard

If there are new changes to the metrics dashboard template that you want to view in an already-existing dashboard. The simplest way to update your dashboard is to delete your prometheus datasource and Trin App metrics dashboard, and re-run the create-dashboard command.

View metrics remotely

Trin metrics on a remote machine can be monitored by listening to the grafana address on a local machine.

On local run:

ssh -N -L <port>:127.0.0.1:<port> <user>@<host>

For example

ssh -N -L 3000:127.0.0.1:3000 username@mycomputer

Then navigate to http://127.0.0.1:3000` in a browser and login with username: admin, password: admin. Then navigate to the trin-app-metrics dashboard.

Metrics setup with docker

1. Install Docker.
2. Run Prometheus, note that you MUST manually set the absolute path to your copy of Trin's etc/prometheus/:

docker run -p 9090:9090 -v /**absolute/path/to/trin/etc/prometheus**:/etc/prometheus --add-host=host.docker.internal:host-gateway prom/prometheus

3. Run Grafana:

docker run -p 3000:3000 --add-host=host.docker.internal:host-gateway grafana/grafana:latest

4. Start your Trin process with:

cargo run -p trin -- --enable-metrics-with-url 0.0.0.0:9100 --web3-http-address http://0.0.0.0:8545 --web3-transport http

• The addresses must be bound to 0.0.0.0, because 127.0.0.1 only allows internal requests to complete, and requests from docker instances are considered external.
• The --enable-metrics-with-url parameter is the address that Trin exports metrics to, and should be equal to the port to which your Prometheus server is targeting at the bottom of prometheus/prometheus.yml
• The --web-transport http will allow Grafana to request routing table information from Trin via JSON-RPC over HTTP

5. From the root of the Trin repo, run cargo run -p trin -- create-dashboard. If you used different ports than detailed in the above steps, or you are not using docker, then this command's defaults will not work. Run the command with the -h flag to see how to provide non-default addresses or credentials.
6. Upon successful dashboard creation, navigate to the dashboard URL that the create-dashboard outputs. Use admin/admin to login.

Gotchas

• If create-dashboard fails with an error, the most likely reason is that it has already been run. From within the Grafana UI, delete the "json-rpc" and "prometheus" datasources and the "trin" dashboard and re-run the command.

• There is a limit on concurrent connections given by the threadpool. At last doc update, that number was 2, but will surely change. If you leave connections open, then new connections will block.

Problems

If you encounter a problem, keep in mind that Trin is under active development. Some issues may be lower on the priority list.

Search for more information

Try searching:

• This book
• The Trin repository issues

Document the problem

If the problem seems new, raise an issue in the Trin repository. Try to record the problem details and include those in the issue. Include details for how someone else might reproduce the problem you have.

FAQ

The following are frequently asked questions or topics that may be of interest to users.

New submissions are welcome, if you had a question and found the answer elsewhere, submit a pull request or an issue describing the question and the answer. These questions will appear in searches in the book and in the trin repository.

Can I rely on Trin to interact with Ethereum?

Not at present. Trin and the Portal Network more broadly are under active development.

Can Trin be used with a VPN?

Trin should be compatible with VPN use, but if you experience difficulty connecting to the network we recommend disabling your VPN.

Can Trin be used over TOR?

No, the Trin uses uTP, which is not supported in the TOR protocol.

Developers

This part of the book is for understanding Trin, and processes around building Trin better.

Where the Trin crates and the Portal Network specification are the source of truth, this section seeks to offer a quicker "key concepts" for getting started.

It seeks to answer questions like:

• What do I need to know about the Portal Network?
• How do the different components of Trin work together?
• What sort of data guarantees are made and how are they achieved?
• What things should a new contributor be mindful of?

Quick setup

This is a single page that aims to cover everything required to get Trin running.

Trin is currently in unstable alpha, and should not be used in production. If you run into any bugs while using Trin, please file an Issue!

Building on Debian Based Systems

Prerequisites

• Rust installation

Building, Testing, and Running

Note: If you use a VPN, you should disable it before running Trin.

Install dependencies (Ubuntu/Debian):

apt install libclang-dev pkg-config build-essential

Environment variables:

# Optional
export RUST_LOG=<error/warn/info/debug/trace>
export TRIN_DATA_PATH=<path-to-data-directory>

Build, test, and run:

git clone https://github.com/ethereum/trin.git
cd trin

# Build
cargo build --workspace

# Run test suite
cargo test --workspace

# Build and run test suite for an individual crate
cargo build -p trin-history
cargo test -p trin-history

# Run
cargo run

Note: You may also pass environment variable values in the same command as the run command. This is especially useful for setting log levels.

RUST_LOG=debug cargo run

View CLI options:

cargo run -- --help

Building on Arch Based Systems

Before starting, update your system.

sudo pacman -Syu

Then, install rust.

sudo pacman -S rustup
rustup install stable

To check that the rust toolchain was successfully installed, run:

rustc --version

You should see something like:

rustc 1.76.0 (82e1608df 2023-12-21)

Next, install the required dependencies:

sudo pacman -S openssl clang pkg-config base-devel llvm git

Now you can build, test and run Trin!

git clone https://github.com/ethereum/trin.git
cd trin

# Build
cargo build --workspace

# Run test suite
cargo test --workspace

# Build and run test suite for an individual crate
cargo build -p trin-history
cargo test -p trin-history

# Run help
cargo run -- --help

# Run Trin with defaults
cargo run

Run locally

Serve portal node web3 access over a different port (such as 8547) using the --web3-http-address flag. The --web3-transport for a local node will be over http (rather than ipc).

RUST_LOG=debug cargo run -- \
    --web3-http-address http://127.0.0.1:8547 \
    --web3-transport http \
    --discovery-port 9009 \
    --bootnodes default \
    --mb 200 \
    --no-stun

Connect to the Portal Network testnet

To immediately connect to the testnet, you can use the --bootnodes default argument to automatically connect with the default Trin bootnodes.

cargo run -- --bootnodes default

To establish a connection with a specific peer, pass in one or more bootnode ENRs. Pass the ENR as the value for the --bootnodes CLI flag.

cargo run -- --bootnodes <bootnode-enr>

Default data directories

• Linux/Unix: $HOME/.local/share/trin
• MacOS: ~/Library/Application Support/trin
• Windows: C:\Users\Username\AppData\Roaming\trin

Using Trin

In some of the following sections, we make use of the web3.py library.

Connect over IPC

In a python shell:

>>> from web3 import Web3
>>> w3 = Web3(Web3.IPCProvider("/tmp/trin-jsonrpc.ipc"))
>>> w3.client_version
'trin 0.0.1-alpha'

To request a custom jsonrpc endpoint, provide the endpoint and array of params. e.g.:

>>> w3.provider.make_request("portal_historyPing", ["enr:-IS4QBz_40AQVBaqlhPIWFwVEphZqPKS3EPso1PwK01nwDMtMCcgK73FppW1C9V_BQRsvWV5QTbT1IYUR-zv8_cnIakDgmlkgnY0gmlwhKRc9_OJc2VjcDI1NmsxoQM-ccaM0TOFvYqC_RY_KhZNhEmWx8zdf6AQALhKyMVyboN1ZHCCE4w", "18446744073709551615"])
{'jsonrpc': '2.0',
 'id': 0,
 'result': {'enrSeq': '3',
  'dataRadius': '115792089237316195423570985008687907853269984665640564039457584007913129639935'}}

See the JSON-RPC API docs for other standard methods that are implemented. You can use the web3.py API to access these.

Connect over HTTP

First launch trin using HTTP as the json-rpc transport protocol:

cargo run -- --web3-transport http

Then, in a python shell:

>>> from web3 import Web3
>>> w3 = Web3(Web3.HTTPProvider("http://127.0.0.1:8545"))
>>> w3.client_version
'trin 0.0.1-alpha'

The client version responds immediately, from the trin client. The block number is retrieved more slowly, by proxying to Infura.

To interact with trin at the lowest possible level, try netcat:

nc -U /tmp/trin-jsonrpc.ipc
{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":83}
{"jsonrpc":"2.0","id":83,"result":"0xb52258"}{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":84}
{"jsonrpc":"2.0","id":84,"result":"0xb52259"}{"jsonrpc":"2.0","id":85,"params":[],"method":"web3_clientVersion"}
{"jsonrpc":"2.0","id":"85","result":"trin 0.0.1-alpha"}
{"jsonrpc":"2.0","id":86,"params":[],"method":"discv5_nodeInfo"}
{"id":86,"jsonrpc":"2.0","result":"enr:-IS4QHK_CnCsQKT-mFTilJ5msHacIJtU91aYe8FhAd_K7G-ACO-FO2GPFOyM7kiphjXMwrNh8Y4mSbN3ufSdBQFzjikBgmlkgnY0gmlwhMCoAMKJc2VjcDI1NmsxoQNa58x56RRRcUeOegry5S4yQvLa6LKlDcbBPHL4H5Oy4oN1ZHCCIyg"}

For something in between, you may use curl to send requests to the HTTP JSON-RPC endpoint.

Developer stories

Trin is under active development. Perhaps you would like to get involved?

The following are some situations that might resonate.

Issue resolver

Someone who tried out Trin and found an issue, then worked out where it was coming from.

Consider making a pull request to fix the issue.

Ecosystem contributor

Trin, and the Portal Network more broadly are perhaps more quiet than other areas of Ethereum development. Maybe you can see yourself helping out somewhere where you can have a meaningful impact.

Researcher

Someone looking into the Ethereum protocol upgrade path, and thinking through the impact of potential upcoming changes.

There are interesting facets to the Portal network still to be determined.

Perhaps you can be tempted by:

• Double batched merkle log accumulators
• Topology of content in distributed hash tables
• Adversarial scenario planning and mitigation

Hobbyist

Someone looking to poke around and run a node on a single board computer or a mobile device. How small is too small?

Rust developer

Someone looking to build something meaningful in Rust, with interesting architecture and crates:

• Cryptography
• Peer to peer networking
• Async runtimes

Goals

Demonstrate feasibility

Implement the Portal Network and demonstrate its use. Starting with subset of the whole and then expanding from there.

Prioritise Sub-protocols

Primary

Get the History sub-protocol working.

• Iron out bugs
• Interop with other clients
• Monitor network to see if it retains data

Secondary

Start work on the State sub-protocol.

• Implementation from the Portal Network specification.

Tertiary

Start work on remaining sub-protocols

• Canonical indices
• Transaction gossip

Progress status

The Portal Network and Trin are under active development so different components may be in varying stages of progress.

• Completed, looking for bugs
• Mid-construction
• Future, planned features

Some methods to get a sense of the state of development are:

• Run trin -- --help and see what flags are available.
• Look at recent closed PRs to see what has just been merged.
• Look at recent issues that have been opened to see what active foci are.
• Look at what examples are shown in the setup and usage guides.
• Run trin in the way you think it might work and see what happens.

Architecture

Trin can be understood a from different perspectives.

• How is code organised?
• How does data flow through trin?
• How is data stored?
• How does testing work?

Workspaces

Trin is a package that can be run:

cargo run -p trin

The trin repository is composed of workspaces that are used by the main Trin package. Their relationship is outlined below.

trin

Code for the trin package is located in ./src.

This crate is responsible for the operation of the Trin node functionality.

• Startup with different configurations via command line arguments
• Starting threads for different important functions such as uTP, Discovery & JSON-RPC.
• These threads perform tasks such as listening for peers or requests from a user.

portalnet

This crate is responsible for the code that defines the main functions and data structures required for the operation of a Trin node. This includes code for:

• Interacting with and managing peers
• Determining content to store and share
• Database management
• Ethereum related data structures

trin-history

This crate is responsible for the History sub-protocol. This means interacting with peers to retrieve and distribute the following:

• Block headers
• Block bodies
• Block receipts

Additionally, it is also responsible for the header accumulator, a structure which provides a mechanism to determine whether a given block hash is part of the canonical set of block hashes.

The crate uses the ethportal-api crate to represent the main data type in this crate: the HistoryContentKey. This struct implements the OverlayContentKey trait, which allows it to be treated as a member of the broader family of OverlayContentKeys.

trin-state

This crate exists mostly as a stub for future work.

This crate is equivalent in function to the trin-history crate, but instead is responsible for the State sub-protocol.

This means that it is responsible for:

• The state of all accounts.
• The state of all contracts.
• The bytecode of all contracts.

Data in the state network is represented as a tries (tree structures). The network uses proofs against these tries to allow Trin nodes to verify the correctness of data.

ethportal-api

This crate seeks to expose the data structures in the Portal Network specification. This includes features such as derived SSZ encoding and convenience functions.

The crate defines traits that may be used across different sub-protocols. For example, the OverlayContentKey may be implemented for content on both the History and State sub-protocols. Thus a function can accept content from both networks via T: OverlayContentKey.

fn handles_content_keys<T: OverlayContentKey>(key: T) {
    // Snip
}

The crate will evolve to provide the types required for the other sub-protocols.

rpc

This crate contains implementations of ethportal-api jsonrpsee server API traits in Trin and interface for running the JSON-RPC server.

utp-testing

Trin uses Micro Transport Protocol (uTP) a UDP based protocol similar to the BitTorrent protocol. This crate can be used to set up clients and servers to test the protocol on a single machine.

ethportal-peertest

Home for integration testing utils used by trin.

Process flow

The following main threads are spawned when Trin is started via ./src/main.rs.

stateDiagram-v2
    trin: trin

    state trin {
        utplistner: UTP listener
        subprotocolhandler: sub-protocol handler
        subprotocolnetworktask: sub-protocol network task
        portaleventshandler: portal events handler
        jsonrpcserver: JSON-RPC server

        main() --> utplistner
        main() --> subprotocolhandler
        main() --> subprotocolnetworktask
        main() --> portaleventshandler
        main() --> jsonrpcserver

    }

Where for each sub-protocol implemented (History, State, Etc.,), a new thread is started.

Here are some of the major components that are called on startup within ./src/lib.rs.

stateDiagram-v2
    collection: configs and services

    state trin {
        main() --> from_cli()
        from_cli() --> run_trin()
        run_trin() --> discovery()
        run_trin() --> utp_listener()
        run_trin() --> header_oracle()
        run_trin() --> portalnet_config
        run_trin() --> storage_config

    }

    portalnet_config --> collection
    storage_config --> collection
    discovery() --> collection
    header_oracle() --> collection
    utp_listener() --> collection

    state portalnet {
        portalnet_config
        storage_config
        discovery()
    }
    state utp {
        utp_listener()
    }
    state validation {
        header_oracle()
    }

Once the initial collection of important configs and services have been aggregated, they are passed to the crates for each sub-protocol (trin-history shown here). The received data structures are then used to start the JSON-RPC server.

An events listener awaits network activity that can be actioned.

stateDiagram-v2
    trinhistory: trin-history
    jsonrpchistory: JSON-RPC History details
    historyhandler: History handler
    collection: configs and services

    state trin {
        collection --> initialize_history_network()
        collection --> HistoryRequestHandler
        initialize_history_network() --> jsonrpchistory
        jsonrpchistory --> launch_jsonrpc_server()
        HistoryRequestHandler --> historyhandler
        collection --> events()
        historyhandler --> events()
    }

    state portalnet {
        events()
    }

    state trinhistory {
        initialize_history_network()
        state jsonrpc {
            HistoryRequestHandler
        }
    }
    state rpc {
        launch_jsonrpc_server()
    }

Then ./portalnet/events.rs is handles events at the level of the Portal Wire Protocol. These are defined messages that are compliant with the Discv5 protocol, and specific to the Portal Network.

Database

The database related code is located in ./portalnet/storage.rs.

There are three main database kinds:

Main content database

This is an SQLite database that stores content data. For a piece of content, this includes the content ID, content key, content value and the size of the content. It makes assessing the size of the database quicker by avoiding the need to repeatedly compute the size of each content.

Memory content database

This uses is an in-memory hashmap to keep content that may not be required for long term storage. An overlay service uses this database when receiving data from a peer as part of Kademlia-related actions. If required, data is later moved to disk in the main content database.

Testing

Testing occurs at different levels of abstraction.

Unit testing

Unit tests are for checking individual data structures and methods. These tests are included within each workspace, at the bottom the file that contains the code being tested. Tests are run by CI tasks on pull requests to the Trin repository.

Integration testing

Tests that involve testing different parts of a crate at the same time are included in a /tests directory within the relevant module or crate. They are also run by CI tasks on pull requests to the Trin repository.

Network simulation

The test-utp crate is part of continuous integration (CI). This sets up client and server insfrastructure on a single machine to test data streaming with simulated packet loss.

Hive

Hive testing runs Trin as a node and challenges it in a peer to peer envorinment. This involves creating a docker image with the Trin binary and passing it to Hive.

Hive itself is a fork of Ethereum hive testing and exists as portal-hive, an external repository (here). It can be started with docker images of other clients for cross-client testing. The nodes are started, fed a small amount of data and then challenged with RPC requests related to that data.

Testing is automated, using docker configurations in the Trin repository to build test Trin and other clients at a regular cadence. Results of the latest test are displayed at https://portal-hive.ethdevops.io/.

Protocols

This section contains summaries of important protocols for the Portal Network. The purpose is to distill important concepts to quickly see how Trin works.

See the relevant specifications for deeper explanations.

JSON-RPC

This is a document for all JSON-RPC API endpoints currently supported by Trin. Trin plans to eventually support the entire Portal Network JSON-RPC API and Ethereum JSON-RPC API.

Currently supported endpoints

Portal Network

The specification for these endpoints can be found here.

• discv5_nodeInfo
• discv5_routingTableInfo
• portal_historyFindContent
• portal_historyFindNodes
• portal_historyGossip
• portal_historyLocalContent
• portal_historyPing
• portal_historyOffer
• portal_historyRecursiveFindContent
• portal_historyStore
• portal_stateFindContent
• portal_stateFindNodes
• portal_stateLocalContent
• portal_stateGossip
• portal_stateOffer
• portal_stateStore
• portal_statePing

Custom Trin JSON-RPC endpoints

The following endpoints are not part of the Portal Network specification and are defined in subsequent sections:

• portal_historyRadius
• portal_historyTraceRecursiveFindContent
• portal_paginateLocalContentKeys
• portal_stateRadius

History Overlay Network

portal_historyRadius

Returns the current data storage radius being used for the History network.

Parameters

None

Returns

• Data storage radius.

Example

{
  "id": 1,
  "jsonrpc": "2.0",
  "result": "18446744073709551615"
}

portal_historyTraceRecursiveFindContent

Same as portal_historyRecursiveFindContent, but will also return a "route" with the content. The "route" contains all of the ENR's contacted during the lookup, and their respective distance to the target content. If the content is available in local storage, the route will contain an empty array.

Parameters

• content_key: Target content key.

Returns

• Target content value, or 0x if the content was not found.
• Network ENRs traversed to find the target content along with their base-2 log distance from the content. If the target content was found in local storage, this will be an empty array.

Example

{
  "id": 1,
  "jsonrpc": "2.0",
  "result": {
	  "content": "0xf90217a06add1c183f1194eb132ca8079197c7f2bc43f644f96bf5ab00a93aa4be499360a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347942a65aca4d5fc5b5c859090a6c34d164135398226a05ae233f6377f0671c612ec2a8bd15c20e428094f2fafc79bead9c55a989294dda064183d9f805f4aecbf532de75e6ad276dc281ba90947ff706beeaecc14eec6f5a059cf53b2f956a914b8360ea6fe271ebe7b10461c736eb16eb1a4121ba3abbb85b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000860710895564a08309a92a832fefd882520884565fc3be98d783010302844765746887676f312e352e31856c696e7578a0c5e99c6e90fbdee5650ff9b6dd41198655872ba32f810de58acb193a954e15898840f1ce50d18d7fdc",
	  "route": [{
            "enr": "enr:-IS4QFoKx0TNU0i-O2Bg7qf4Ohypb14-jb7Osuotnm74UVgfXjF4ohvk55ijI_UiOyStfLjpWUZsjugayK-k8WFxhzkBgmlkgnY0gmlwhISdQv2Jc2VjcDI1NmsxoQOuY9X8mZHUYbjqVTV4dXA4LYZarOIxnhcAqb40vMU9-YN1ZHCCZoU",
            "distance": 256
	  }]
  }
}

State Overlay Network

portal_stateRadius

Returns the current data storage radius being used for the State network.

Parameters

None

Returns

• Data storage radius.

Example

{
  "id": 1,
  "jsonrpc": "2.0",
  "result": "18446744073709551615"
}

General

portal_paginateLocalContentKeys

Return a paginated list of all of the content keys (from every subnetwork) currently available in local storage.

Parameters

• offset: The number of records that need to be skipped.
• limit: Number of entries to return.

Returns

• content_keys: List of content keys.
• total_entries: Total number of content keys in local storage.

Example

{
  "id": 1,
  "jsonrpc": "2.0",
  "result": {
    "content_keys": ["0x0055b11b918355b1ef9c5db810302ebad0bf2544255b530cdce90674d5887bb286"],
    "total_entries": 1
  }
}

Core concepts

This section contains specific concepts that are common, important or that have an interesting facet to understand.

Finding peers

If a peer is in a network behind a NAT (Network Address Translation) table, the process for finding a peer is more complicated.

These diagrams are intended as a rough-guide.

Non-NAT simple case

The bootnode can gossip to Charlie who can then directly contact Alice.

sequenceDiagram
    Alice IP1 PORT1-->>Bootnode: Hello (ENR with no IP)
    Bootnode-->>Alice IP1 PORT1: Hi, I notice your address is <IP1>:<PORT1>
    Alice IP1 PORT1-->>Alice IP1 PORT1: Updates ENR (<IP1>:<PORT1>)
    Bootnode-->>Charlie: Meet Alice (ENR with <IP1>:<PORT1>)
    Charlie->>Alice IP1 PORT1: Hello Alice at <IP1>:<PORT1>
    Alice IP1 PORT1->>Charlie: Hello (ENR <IP1>:<PORT1>)

NAT problem

The bootnode can gossip to Charlie, but Charlie is a stranger from the NAT's perspective. It doesn't know who on the internal network is the recipient.

• The NAT remembers who it has spoken to.
• Messages from the bootnode are expected.
• Messages from Charlie are not expected, and its not clear who they are for. Perhaps the smart fridge?

sequenceDiagram
    Alice IP1 PORT1-->>NAT IP2 PORT2: Hello bootnode (ENR with no IP)
    Note right of NAT IP2 PORT2: Stores Bootnode
    NAT IP2 PORT2-->>NAT IP2 PORT2: Maps from internal IP
    NAT IP2 PORT2-->>Bootnode: Hello bootnode (ENR with no IP)
    Bootnode-->>NAT IP2 PORT2: Hi, I notice your address is <IP2>:<PORT2>
    NAT IP2 PORT2-->>NAT IP2 PORT2: Maps to internal IP
    NAT IP2 PORT2-->>Alice IP1 PORT1: Hi, I notice your address is <IP2>:<PORT2>
    Alice IP1 PORT1-->>Alice IP1 PORT1: Updates ENR (<IP2>:<PORT2>)
    Alice IP1 PORT1-->>NAT IP2 PORT2: Thanks bootnode (ENR with <IP2>:<PORT2>)
    NAT IP2 PORT2-->>Bootnode: Thanks boodnode (ENR with <IP2>:<PORT2>)
    Bootnode-->>Charlie: Meet Alice (ENR with <IP2>:<PORT2>)
    Charlie->>NAT IP2 PORT2: Hello Alice at <IP2>:<PORT2>
    Note right of NAT IP2 PORT2: No map on record. Who is this for?
    Note right of Charlie: Hmm Alice didn't respond.

The NAT solution

If Alice knows she is behind a NAT, she can pass a message which goes:

"I'm behind a NAT. Send your requests via peers and I'll reach out to you."

• The bootnode gossips to Charlie
• Charlie sees "NAT" in Alices ENR
• Charlie asks the bootnode to introduce him to Alice
• Alice reaches out to Charlie
• The NAT now has a mapping for Charlie-Alice messages.

Part 1: NAT detection

Alice can suspect that she is behind a NAT probabilistically. If 2 minutes after connecting with a bootnode, no strangers (like Charlie) have reached out, a NAT is likely.

sequenceDiagram
    Alice IP1 PORT1-->>NAT IP2 PORT2: Hello bootnode (ENR with no IP)
    Note right of NAT IP2 PORT2: Stores Bootnode
    NAT IP2 PORT2-->>NAT IP2 PORT2: Maps from internal IP
    NAT IP2 PORT2-->>Bootnode: Hello bootnode (ENR with no IP)
    Bootnode-->>NAT IP2 PORT2: Hi, I notice your address is <IP2>:<PORT2>
    NAT IP2 PORT2-->>NAT IP2 PORT2: Maps to internal IP
    NAT IP2 PORT2-->>Alice IP1 PORT1: Hi, I notice your address is <IP2>:<PORT2>
    Alice IP1 PORT1-->>Alice IP1 PORT1: Updates ENR (<IP2>:<PORT2>)
    Alice IP1 PORT1-->>NAT IP2 PORT2: Thanks bootnode (ENR with <IP2>:<PORT2>)
    NAT IP2 PORT2-->>Bootnode: Thanks boodnode (ENR with <IP2>:<PORT2>)
    Note right of Alice IP1 PORT1: ... Hmm no strangers. Must be a NAT.

Part 2: NAT communication

Alice can put "NAT" in her ENR. Now when Charlie tries to get in touch, he knows to go via a peer.

Continued from above, skipping Charlie's failed attempt to contact Alice directly.

sequenceDiagram
    Note right of Alice IP1 PORT1: ... Hmm no strangers. Must be a NAT.
    Alice IP1 PORT1-->>NAT IP2 PORT2: Update: NAT (ENR with NAT <IP2>:<PORT2>)
    NAT IP2 PORT2-->>Bootnode: Update: NAT (ENR with NAT <IP2>:<PORT2>)
    Bootnode-->>Charlie: Meet Alice (ENR with NAT <IP2>:<PORT2>)
    Charlie->>Bootnode: Hello Alice (From Charlie ENR(<charlie>))
    Note right of Bootnode: To Alice via Bootnode
    Bootnode->>NAT IP2 PORT2: Hello Alice (From Charlie ENR(<charlie>))
    NAT IP2 PORT2-->>NAT IP2 PORT2: Maps to internal IP
    NAT IP2 PORT2-->>Alice IP1 PORT1:  Hello Alice (From Charlie ENR(<charlie>))
    Alice IP1 PORT1-->>NAT IP2 PORT2: Hello Charlie (ENR with NAT <IP2>:<PORT2>)
    Note right of NAT IP2 PORT2: Stores Charlie
    NAT IP2 PORT2-->>NAT IP2 PORT2: Maps from internal IP
    NAT IP2 PORT2-->>Charlie: Hello Charlie (ENR with NAT <IP2>:<PORT2>)
    Charlie-->>NAT IP2 PORT2: Hi Alice
    NAT IP2 PORT2-->>NAT IP2 PORT2: Maps to internal IP
    Note right of NAT IP2 PORT2: Finally has a mapping for Charlie!
    NAT IP2 PORT2-->>Alice IP1 PORT1: Hello Alice

Chain tip

A Trin node can serve information about the chain tip, such as the latest block number. A Trin node knows about the beacon chain protocol that is creating the chain tip.

By listening to activity on the beacon chain network, it can follow the activities of members of the sync committee. If a certain fraction of the sync committee have signed off on a certain beacon block, the Trin node can be confident that this is likely to be the chain tip.

Beacon blocks contain references to Ethereum blocks, and so the node can see the tip of the Execution chain.

Cryptographic accumulator

A cryptographic accumulator is a structure that allows verification of a specific block header in the past is part of the canonical chain.

The History sub-protocol is responsible for accumulator-related data.

An accumulator has been constructed for the Portal Network, because it is too burdensome to keep all the headers on disk. This applies to pre-merge blocks. For post-merge blocks, the Beacon Chain already maintains an accumulator that Trin can use via a Beacon Chain light client.

Canonicality

A block can be valid but not canonical if it is an Uncle. Blocks A-F are canonical, with F being the latest block.

While Uncle_1 may have a valid block difficulty and parent, it was not built upon.

flowchart RL
    Uncle_3-.->D
    F--->E
    E--->D
    Uncle_4-.->D
    Uncle_1-.->C
    D--->C
    Uncle_2-.->C;
    C--->B;
    B--->A;

If a Trin node is presented with such a block, it can check the accumulator, which only processes non-uncle blocks A-F.

Tip knowledge

First, the most recent block hash at the tip of the accumulator must be known.

This is easy, as the accumulator only needs to cover pre-merge blocks. The final pre-merge block (last Proof of Work block) hash is known and never needs to be updated.

Proofs

A Merkle proof can be constructed for any given historical block hash. The proof asserts that a given hash (from a peer) is part of the accumulator (valid based on knowledge of the current chain tip).

A proof cannot be constructed for any other sort of block (Uncle block, fabricated block).

Accumulator construction

The accumulator is specifically a double-batched merkle log accumulator.

First historical blocks are processed in batches called Epochs (unrelated to the concept of a 32-slot epoch in the Beacon Chain).

The accumulator constructor consists of two lists:

• One cache for holding blocks (header and difficulty).
• One final store (Master Accumulator) that the cache roots are added to.

flowchart TD
    Genesis-->new_epoch[Start new Epoch Accumulator]
    new_epoch-->append
    append[Append block header and difficulty to Epoch Accumulator]
    append--> epoch_done{Done 8192 yet?}
    epoch_done -.->|Yes| get_epoch_root
    epoch_done -->|No| PoS{PoS reached?}
    PoS --> |No| append
    PoS -.-> |Yes| finished[Finished]
    add_root[Append root to Master Accumulator] -.-> new_epoch
    get_epoch_root[Compute hash tree root of epoch] -.-> add_root
    finished -.-> save[Save incomplete final epoch and Master Accumulator]

Thus the final output is a list of roots called the Master Accumulator.

Constructing proofs

If you have a block and you know the block number, then you know which epoch root is relevant. You also know which part of the epoch it came from. That is, you know the index of the leaf in the Merkle tree.

With the root of the tree, the index and the data (the block hash in question), a proof can be constructed showing that this leaf was part of the tree.

Proof use

A proof can be to a peer alongside the data. That way, a peer can quickly and check that the data is canonical.

Accumulator distribution

The Accumulator is built once and then distributed in Trin (and other clients). It does not change over time and so can be incorporated into the trin-validation (./trin-validation/src/assets) and included in binary releases.

The History network contains individual epoch hashes from the Master Accumulator and refers to them with the terms: epoch_accumulator and epoch_accumulator_key (includes selector). See the History sub-protocol section of the Portal Network spec.

Master accumulator details

The Master Accumulator consists of:

• 1895 complete epoch roots
• 1 incomplete epoch root (a partial epoch with 5362 records (block headers))

epoch,index
8191,0x5ec1ffb8c3b146f42606c74ced973dc16ec5a107c0345858c343fc94780b4218 // first epoch
16383,0xa5364e9a9bc513c4601f0d62e6b46dbdedf3200bbfae54d6350f46f2c7a01938
...
15523839,0x804008940c025a4e8a00ea42a659b484ba32c14dff133e9d3b7bf3685c1e54de // penultimate epoch (full)
15532031,0x3f81607c8cb3f0448a11cab8df0e504b605581f4891a9a35bd9c0dd37a71834f // final epoch (incomplete)

Final PoW block: 15537394

The hash tree root of the Master Accumulator is:

0x8eac399e24480dce3cfe06f4bdecba51c6e5d0c46200e3e8611a0b44a3a69ff9

Bridge

Blocks are produced by Ethereum Execution clients which use a different network to Portal Network nodes. A Bridge node is responsible for taking data from the external network and passing it to the Portal Network.

flowchart LR
    eth[Ethereum Execution node]-->bridge[Portal Network Bridge node]
    bridge-->portal[Portal network node]

This operates as follows:

sequenceDiagram
    Bridge-->>Execution: eth_getBlock
    Execution-->>Bridge: block
    Bridge-->>Portal: block

Currently the bridge functionality exists as a separate executable under portal-bridge.

Archive nodes

A Portal Network node is not an archival node. This page explores the reason for this and some considerations on the topic.

An archive node is one that can know the history at a certain block in the past.

A non-archive node has this information until a block is 128 blocks old. After this point the data is forgotten.

Old state

Archive nodes store old states

• What was the balance of token x at block y?
• What was in storage slot x at block y?

Old traces

Archive nodes store old traces. This means that they can re-execute old transactions and show everything that the EVM did.

• What events were emitted during transaction x?
• How much gas did transaction x use?

Requirements

Consider an archive node that is going to trace the 100th transaction in an old block.

• The transaction may call a contract, which may in turn call another contract (etc., ). The state of the contracts must be known (balance, nonce, bytecode, storage)
• The transaction may reference the hash of a preceding block (up to depth of 256 blocks)
• The transaction may modify state that has already been modified in the preceding 99 transactions.

Would an Archive sub-protocol be of use?

Not for sequential data analysis

Archival nodes are great for data science because they allow traversing a large number of sequential blocks and tracking changes over time.

A portal node would not be suited for this activity because it requires sequential blocks rather than possession of data based on the nodes ID. Hence a Portal Node has a disperse subset of content and would need to ask peers for data for sequential blocks. Asking for all sequential blocks would cause an infeasible burden on peers.

Possibly for personal wallet history

A user with access to an index of address appearances (such as the Unchained Index) could make queries about their historical transactions. This could be for a wallet, multisig contract or any contract.

After retrieving the traces for these transactions, they could be used to create a display of activity. E.g., A graph of token balances changing over time, or a log of on-chain activity (trades, loans, transfers, NFT activity).

Could an Archive sub-protocol exist?

It is not impossible. However, the goal of the Portal Network is to provide the function of a non-tracing node. Some considerations are explored below.

Intra-block state

To trace the last transaction in a block, all preceding transaction final states must be known. Hence single request for a transaction trace could result in requiring many transactions in a single block to be obtained. This applies to popular contracts that appear frequently in a block (e.g., exchanges, and popular tokens).

Consequence of a request for a transaction at the end of a block involving popular contracts:

• It would be very slow to get a response
• It could be used as a denial of service (DoS) attack on the network. For instance, by finding the final transactions in blocks and requesting them from different nodes.

Nested contract calls

A contract could start a chain of nested calls to other contracts. If a node does not have the state of these contracts, it would have to request them. Hence, the time to trace such a transaction would be very slow. Every nested call would take the time that a single Portal Network request takes.

Consequences of a request for a transaction with deeply nested contract calls:

• It would be very slow to get a response
• It could be used as a denial of service (DoS) attack on the network. For instance, by finding many nested transactions and requesting them from different nodes.

Duplication of data

If Archive was a sub-protocol there may be some data that is required to be duplicated on the History or State sub-protocols. This implies that the sub-protocol is inefficient with respect to disk space but may not be a significant problem.

Medium-sized portal nodes

There is a always a moderate amount of interest in archive nodes, for many parties find historical Ethereum data valuable. As archive nodes require minimum ~2TB of storage, many people choose not to run one.

Perhaps there is a large enough appetite to run a "medium-sized portal archive node", such that many users contribute ~100GB. In this scenario, the DoS attacks are reduced as these medium-sized nodes would cause less amplification of network traffic.

Appetite for lags

If the desire for the results of an archive node are large enough, applications and users could be tolerant of slow lookup times. For example, a wallet connected to a portal archive node could display current wallet state quickly, but under a "history" tab could show: "performing deep search... Estimated time 24 hours". Once the information has been retrieved it could then be stored for fast access.

Contributor guidelines

These guidelines are heavily influenced by the Snake-Charmer Tactical Manual. While the manual is written with a focus on Python projects, there is tons of relevant information in there for how to effectively contribute to open-source projects, and it's recommended that you look through the manual before contributing.

Rust

Trin is written in Rust. This section includes guidelines for Rust-specific patterns and principles.

Comments

Any datatype of significance should have an accompanying comment briefly describing its role and responsibilities. Comments are an extremely valuable tool in open-source projects with many different contributors, and can greatly improve development speed. Explain your assumptions clearly so others don't need to dig through the code.

• Rust doc comments are the most best way to comment your code.

Imports

• In *.rs files, imports should be split into 3 groups src and separated by a single line. Within a single group, imported items should be sorted alphabetically.
  • Imports from 'std'
  • Imports from external crates
  • Imports from crates within trin
• Alphabetize imports in Cargo.toml

Logging

• All logging should be done with the log library and not println!() statements.
• Appropriate log levels (debug, warn, info, etc.) should be used with respect to their content.
• Log statements should be declarative, useful, succinct and formatted for readability.

Bad:

Oct 25 23:42:11.079 DEBUG trin_core::portalnet::events: Got discv5 event TalkRequest(TalkRequest { id: RequestId([226, 151, 109, 239, 115, 223, 116, 109]), node_address: NodeAddress { socket_addr: 127.0.0.1:4568, node_id: NodeId { raw: [5, 208, 240, 167, 153, 116, 216, 224, 160, 101, 80, 229, 154, 206, 113, 239, 182, 109, 181, 137, 16, 96, 251, 63, 85, 223, 235, 208, 3, 242, 175, 11] } }, protocol: [115, 116, 97, 116, 101], body: [1, 1, 0, 0, 0, 0, 0, 0, 0, 12, 0, 0, 0, 1, 0, 0, 0, 255, 255, 255, 255, 255, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 36, 0, 0, 0, 0, 0, 0, 0], sender: Some(UnboundedSender { chan: Tx { inner: Chan { tx: Tx { block_tail: 0x55c4fe611290, tail_position: 1 }, semaphore: 0, rx_waker: AtomicWaker, tx_count: 2, rx_fields: "..." } } }) })

Good:

Oct 25 23:43:02.373 DEBUG trin_core::portalnet::overlay: Received Ping(enr_seq=1, radius=18446744073709551615)

Error handling

• Handle errors. Naked .unwrap()s aren't allowed, except for in unit tests. Exceptions must be accompanied by a note justifying usage.
  • In most cases where an exception can be made (E.g., parsing a static value) .expect() with a relevant message should be used over a naked unwrap.
• Write descriptive error messages that give context of the problem that occurred. Error messages should be unique, to aid with debugging.
• Meaningful error types should be used in place of Result< _, String>.
  • General errors should use the anyhow crate.
  • Custom / typed errors should derive from the std::error::Error trait. The thiserror crate provides a useful macro to simplify creating custom error types.

Style

Clone

Minimize the amount of .clone()s used. Cloning can be a useful mechanism, but should be used with discretion. When leaned upon excessively to satisfy the borrow checker it can lead to unintended consequences.

String interpolation

Use interpolated string formatting when possible.

• Do format!("words: {var:?}") not format!("words: {:?}", var)

Git

This section covers guidelines and common scenarios encountered with using git and github for Trin development.

Commit messages

Commit Hygiene

We do not have any stringent requirements on how you commit your work, however you should work towards the following with your git habits.

Logical Commits

This means that each commit contains one logical change to the code. For example:

• commit A introduces new API
• commit B deprecates or removes the old API being replaced.
• commit C modifies the configuration for CI.

This approach is sometimes easier to do after all of the code has been written. Once things are complete, you can git reset master to unstage all of the changes you've made, and then re-commit them in small chunks using git add -p.

Commit Messages

We use conventional commits for our commit messages. This means that your commit messages should be of the form:

<type>[optional scope]: <description>

To learn more about conventional commits please check out the conventional commits website.

Examples:

• fix: Update metrics strategy to support multiple subnetworks
• refactor(light-client): Refactor light-client crate to use ethportal-api consensus types
• feat(rpc): Return header to eth_getBlockByHash

One way to test whether you have it right is to complete the following sentence.

If you apply this commit it will ________________.

Submodules

This project uses Git Submodules. If you just cloned the project, be sure to run:

$ git submodule update --init

This page provides short overview of most common use cases.

Pulling in Upstream Changes from the Submodule Remote

You want to do this when the remote version of submodule is updated. The simplest way to resolve this is to run:

$ git submodule update --remote --rebase

If you modified your local submodule, you might want to use different flags.

If you run git status, you should see that submodule is updated. Commit and push the changes so others can use the same version.

Pulling Upstream Changes from the Project Remote

If somebody else updated the submodule and you pulled the changes, you have to update your local clone as well. The message "Submodules changed but not updated" will show when running git status. To update local submodule, run:

$ git submodule update --init

Rebasing

You should be using git rebase when there are upstream changes that you need in your branch. You should not use git merge to pull in these changes.

Release notes

We use conventional commits to generate release notes.

Check the commits guide for more information on how to write commit messages.

Pull requests

We are a distributed team. The primary way we communicate about our code is through github via pull requests.

• When you start work on something you should have a pull request opened that same day.
• Mark unfinished pull requests with the "Work in Progress" label.
• Before submitting a pr for review, you should run the following commands locally and make sure they are passing, otherwise CI will raise an error.
  • cargo +nightly fmt --all -- --check and cargo clippy --all --all-targets --all-features -- --deny warnings for linting checks
  • RUSTFLAGS='-D warnings' cargo test --workspace to run all tests
• Pull requests should always be reviewed by another member of the team prior to being merged.
  • Obvious exceptions include very small pull requests.
  • Less obvious examples include things like time-sensitive fixes.
• You should not expect feedback on a pull request which is not passing CI.
  • Obvious exceptions include soliciting high-level feedback on your approach.

Large pull requests (above 200-400 lines of code changed) cannot be effectively reviewed. If your pull request exceeds this threshold you should make every effort to divide it into smaller pieces.

You as the person opening the pull request should assign a reviewer.

Code review

Reviewing

Every team member is responsible for reviewing code. The designations :speech_balloon:, :heavy_check_mark:, and :x: should be left by a reviewer as follows:

• :speech_balloon: (Comment) should be used when there is not yet an opinion on overall validity of complete PR, for example:
  • comments from a partial review
  • comments from a complete review on a Work in Progress PR
  • questions or non-specific concerns, where the answer might trigger an expected change before merging
• :heavy_check_mark: (Approve) should be used when the reviewer would consider it acceptable for the contributor to merge, after addressing all the comments. For example:
  • style nitpick comments
  • compliments or highlights of excellent patterns ("addressing" might be in the form of a reply that defines scenarios where the pattern could be used more in the code, or a simple :+1:)
  • a specific concern, where multiple reasonable solutions can adequately resolve the concern
  • a Work in Progress PR that is far enough along
• :x: (Request changes) should be used when the reviewer considers it unacceptable to merge without another review of changes that address the comments. For example:
  • a specific concern, without a satisfactory solution in mind
  • a specific concern with a satisfactory solution provided, but alternative solutions may be unacceptable
  • any concern with significant subtleties

Responding

Contributors should react to reviews as follows:

• :x: if any review is marked as "Request changes":
  • make changes and/or request clarification
  • should not merge until reviewer has reviewed again and changed the status
• (none) if there are no reviews, contributor should not merge.
• :speech_balloon: if all reviews are comments, then address the comments. Otherwise, treat as if no one has reviewed the PR.
• :heavy_check_mark: if at least one review is Approved, contributor should do these things before merging:
  • make requested changes
  • if any concern is unclear in any way, ask the reviewer for clarification before merging
  • solve a concern with suggested, or alternative, solution
  • if the reviewer's concern is clearly a misunderstanding, explain and merge. Contributor should be on the lookout for followup clarifications on the closed PR
  • if the contributor simply disagrees with the concern, it would be best to communicate with the reviewer before merging
  • if the PR is approved as a work-in-progress: consider reducing the scope of the PR to roughly the current state, and merging. (multiple smaller PRs is better than one big one)

It is also recommended to use the emoji responses to signal agreement or that you've seen a comment and will address it rather than replying. This reduces github inbox spam.

Everyone is free to review any pull request.

Recommended Reading:

• How to Do Code Reviews Like a Human

Fetching a pull request

We often want or need to run code that someone proposes in a PR. Typically this involves adding the remote of the PR author locally and then fetching their branches.

Example:

git remote add someone https://github.com/someone/reponame.git
git fetch someone
git checkout someone/branch-name

With an increasing number of different contributors this workflow becomes tedious. Luckily, there's a little trick that greatly improves the workflow as it lets us pull down any PR without adding another remote.

To do this, we just have to add the following line in the [remote "origin"] section of the .git/config file in our local repository.

fetch = +refs/pull/*/head:refs/remotes/origin/pr/*

Then, checking out a PR locally becomes as easy as:

git fetch origin
git checkout origin/pr/<number>

Replace origin ☝ with the actual name (e.g. upstream) that we use for the remote that we want to fetch PRs from.

Notice that fetching PRs this way is read-only which means that in case we do want to contribute back to the PR (and the author has this enabled), we would still need to add their remote explicitly.

Merging

Once your pull request has been Approved it may be merged at your discretion. In most cases responsibility for merging is left to the person who opened the pull request, however for simple pull requests it is fine for anyone to merge.

If substantive changes are made after the pull request has been marked Approved you should ask for an additional round of review.

Team Rotation - aka "Flamingo" 🦩

Recently, we started a new process. It is evolving, so this doc will evolve with it.

Every Monday, a full-time member of trin rotates into a maintenance position. The person in that role is called the "Flamingo" (truly amazing animals).

Some responsibilities for the Flamingo are:

• monitor glados & hive to watch for regressions; investigate & resolve them
• answer questions from other Portal teams
• run releases & deployments
• review PRs that are stuck
• add documentation & build tools
• fun integrations, like Discord bots

The idea is to set aside normal coding progress for the week, to take care of these important and often overlooked tasks.

Responsibilities

There are some daily tasks, and some transition tasks: Kickoff on Monday & Handoff on Friday.

Unlike your code-heavy weeks, this can be an interruption-heavy week. Although it's often good practice to not get distracted by inbound messages while coding, this week is the opposite: as soon as you see new messages (like on Discord), jump into it.

Start with Kickoff on Monday morning:

Kickoff

At the end of the previous week, you will get the Flamingo notes from the previous Flamingo. They will include incidents from the prior weeks and any in-progress tasks. This will be crucial to get a quick understanding of the state of the network and any in-progress tasks to resume.

Read through the notes, and then generate the new checklist for the week, by creating a note from this template.

Link the generated checklist into the Flamingo notes for your week. Make sure your status is "online" in Discord. Make sure you're tagged under the trin-flamingo role. Put on your favorite pink shirt. Watch a silly flamingo video. Fly.

First

Read through the "Setup" section of the Deployment Instructions and follow the steps to make sure that your PGP and SSH keys are in place and ready for a deployment.

Daily

Checklist

Every day, go down the checklist for that day of the week, resolve items and check them off. If you think of new daily things to add to the checklist, comment on the template to suggest it.

As long as there aren't any major incidents, you should finish the checklist with plenty of time left in your day. See Maintenance Inspiration for what to do for the rest of the day.

Maintenance Inspiration

When you get to the end of your checklist, here are ideas for what to work on next:

• Respond to Github Participating Notifications
• Review PRs that have been stuck for >24 hours
• Find a Flamingo Issue that seems promising, and assign it to yourself (and the project dashboard)
• grep code for TODOs. For each one you find:
  • write up issue to explain what needs to be done, and a plan
  • link to the TODO in the code, and any relevant context
  • label issue as Flamingo. If it is not urgent and a good fit, add "Good First Issue"
  • Post a PR to remove the TODO from the code, in favor of the issue.
  • git grep -iE "(todo)|(fixme)"
• Look through all open trin issues
  • Close outdated issues, with a short explanation
  • If appropriate, add a Flamingo tag
  • Sort by "Least Recently Updated" and tackle the most stale issues, until the stale issue bot is active & we've caught up to date with 0 stale issues.
• Write new portal-hive tests
  • The first step is to make an issue with a list of test ideas, or take over this one
  • Add new portal-hive test ideas to the list
  • Claim one of the ideas of the list to work on
• Respond to all trin repo activity
• Add new integration tools (eg~ portal-hive -> Discord bot?)
• Scratch your own itch! What do you wish was easier? Now is the time to build it.

Be sure to update the week's notes as you go, adding: notable incidents, summaries of what you are working on, and ideas for future work.

Handoff

Before handing off the notes on Friday, be sure to read through the notes to cleanly summarize the week's activities. Link to any helpful issues, and delete any text that is only meaningful to you.

Double check the Flamingo schedule and make sure you're available for your next rotation. If not, please switch with somebody asap.

Think through: what kinds of things do you wish were on the notes that you got on Monday?

After a while, everyone should have the Flamingo Notes link. Sending the link anyway helps accomplish a few things:

1. Signal to the next Flamingo that you're done writing up the notes
2. Give the next Flamingo a reminder that their turn is coming up
3. A failsafe to make sure we didn't forget to pick a Flamingo for the following week

Especially while we continue to develop the procedure, try to be available the following Monday to help the subsequent Flamingo transition in, and get started.

Being the Announcer

The Flamingo is often the first to notice downtime. Whether it's something that affects users or developers, announce the downtime and link to status updates as soon as possible. Also, announce when the downtime is over.

Not Flamingo

If you're not Flamingo currently, there are some situations where it is important to ping the Flamingo in Discord with @flamingo-trin. For example:

• You notice the network behaving abnormally
• You notice any of our tooling behaving abnormally (github, Circle, glados, etc)
• You have a PR that has had no review activity after asking more than 24 hours ago
• You are doing anything that affects other people, like:
  • planned downtime
  • a team-wide configuration change in CircleCI
  • deploying a new tool

Naturally, before writing it in chat, look for a pre-existing announcement from the Flamingo. Maybe they already figured it out, which means they have already announced it, and we don't need the duplicate message.

Build Instructions

The following are guides for building Trin on different platforms. Other methods may be used as appropriate.

• Mac OS
• Linux
• Raspberry Pi
• Windows

Mac Os

Clone trin and run.

$ cd ~
$ git clone https://github.com/ethereum/trin.git
$ cd trin
$ cargo run -p trin --release

Linux

Installing Trin on Arch Linux

There's an AUR package for Trin. You can install it with your favorite AUR helper, e.g. yay:

yay -S trin-git

Trin as a service on Ubuntu

These steps are for setting up a Trin node as a service on Ubuntu.

Installation

$ sudo apt install libclang-dev pkg-config build-essential

Install Trin:

Tip: If you intend to submit code changes to trin, first fork the repo and then clone that url.

$ cd ~
$ git clone https://github.com/ethereum/trin.git
$ cd trin
$ cargo build --workspace --release

Now the executable is located in trin/target/release and can be called by systemd. Move that binary to the standard location for binaries:

$ sudo cp -a ~/trin/target/release/trin /usr/local/bin/trin

Tip: If you make changes to these steps, keep a record for future reference.

Make a new user for the Trin service:

$ sudo useradd --no-create-home --shell /bin/false trin

Make a directory for Trin data and give the Trin user permission to access it:

$ sudo mkdir -p /var/lib/trin
$ sudo chown -R trin:trin /var/lib/trin

Check that the binary works:

$ /usr/local/bin/trin --version

Example response:

> Launching trin
> trin 0.0.1

Configuration

Before setting up the service, look at the flags that can be set when starting Trin:

$ /usr/local/bin/trin --help

Some selected flags are described below.

Optional flag for database size

--mb 200. Trin lets you control how much storage the node takes up (e.g., 200MB). The default is 100 megabytes and can be changed.

Optional flag for no connection to external server

--no-stun. A third party server connection is configured by default to assist in testing. This is a Session Traversal Utilities for NAT (STUN) server and may be disabled a flag. The docs state: "Do not use STUN to determine an external IP. Leaves ENR entry for IP blank. Some users report better connections over VPN."

Optional flags for conflicting nodes

The discovery and JSON-RPC ports may conflict with an existing an Ethereum client or other software on the same machine.

--discovery-port <port>. The default port is 9009. Pick something else if in use.

--web3-http-address <ip_address>:<port>. If an Ethereum execution client is already running, it may be using the default port 8545. The localhost IP address (127.0.0.1) is recommended here.

--web3-transport http. If a new http port is specified using --web3-http-address (as above), the transport must also be changed to http from the default (ipc).

To pick a new port, select a number in the range 1024–49151 and test if it is in use (no response indicates it is ok to use):

$ sudo ss -tulpn | grep ':9008'

Optional flag to enable websocket rpc

--ws. This will allow you to run a websocket on port 8546 --ws --ws-port 3334. A custom websocket port can be configured like this

Create the node service

Create a service to run the Trin node:

$ sudo nano /etc/systemd/system/trin.service

Paste the following, modifying flags as appropriate:

Tip: Note that backslash is needed if starting a flag on a new line.

[Unit]
Description=Trin Portal Network client
After=network.target
Wants=network.target
[Service]
User=trin
Group=trin
Type=simple
Restart=always
RestartSec=5
ExecStart=/usr/local/bin/trin \
    --discovery-port 9008 \
    --web3-http-address 127.0.0.1:8543 \
    --web3-transport http \
    --bootnodes default \
    --mb 200 \
    --no-stun
[Install]
WantedBy=default.target

CTRL-X then CTRL-Y to exit and save.

Tip: Note that we are not using the default discovery (9009) and HTTP (8545) ports. This is done on purpose with the assumption that we will run Trin from the source code as well. See section below.

Add environment variables

The environment variables are going in a different file so they are not accidentally copy-pasted to public places. Create the override.conf file, which will be placed in a new trin.service.d directory beside the trin.service file:

$ sudo systemctl edit trin

Open the file:

$ sudo nano /etc/systemd/system/trin.service.d/override.conf

Tip: The 'info' level of logs is a good starting value.

[Service]
# (optional) Rust log level: <error/warn/info/debug/trace>
Environment="RUST_LOG=info"
# (optional) This flag sets the data directory to the location we created earlier.
Environment="TRIN_DATA_PATH=/var/lib/trin"

Configure firewall

Ensure that the discovery port (custom or default 9009) is not blocked by the firewall:

$ sudo ufw allow 9009

Check the configuration:

$ sudo ufw status numbered

Tip: use sudo ufw delete <number> to remove a particular rule.

Start the service

Start the Trin node service and enable it to start on reboot:

$ sudo systemctl daemon-reload
$ sudo systemctl start trin
$ sudo systemctl status trin
$ sudo systemctl enable trin

Follow Trin's logs:

$ sudo journalctl -fu trin

CTRL-C to to exit.

Logs can be searched for an "exact phrase":

$ grep "trin" /var/log/syslog | grep "exact phrase"

To stop Trin and disable it from starting on reboot:

$ sudo systemctl stop trin
$ sudo systemctl disable trin

Keeping service up to date

To get upstream updates, sync your fork with upstream on Github. To move any changes from the codebase to the service, rebuild and move the binary as before:

$ git pull
$ cd trin
$ cargo build --workspace --release
$ sudo systemctl stop trin
$ sudo cp -a ~/trin/target/release/trin /usr/local/bin/trin

Restart the service to use the new binary:

$ sudo systemctl daemon-reload
$ sudo systemctl start trin

Trin from the source

See getting started notes for more tips including setting environment variables during testing.

Tip: If Trin service is using non-default ports (as suggested earlier), you don't have to set ports now. Either way, make sure ports are not already in use.

$ cargo test --workspace
$ cargo run -- --discovery-port 9009 \
    --web3-http-address 127.0.0.1:8545 \
    --web3-transport http \
    --bootnodes default \
    --mb 200 \
    --no-stun

Raspberry Pi

Not yet attempted, but experiments are encouraged.

Windows

These are instructions for Native and cross compiling Windows builds

Native compilation of Windows

If you don't already have Rust install it

$ winget install Rustlang.Rustup

Install clang/llvm as it is required to compile c-kzg

$ winget install LLVM.LLVM

Add Rust's msvc target

$ rustup target add x86_64-pc-windows-msvc

Install target toolchain

$ rustup toolchain install stable-x86_64-pc-windows-msvc

Build Trin

$ cargo build -p trin

Cross-compilation for Ubuntu compiling to Windows

This is assuming you already have rust installed on Linux

Install required dependencies

$ sudo apt update
$ sudo apt upgrade
$ sudo apt install git clang g++-mingw-w64-x86-64-posix

Clone trin and build.

$ git clone https://github.com/ethereum/trin.git
$ cd trin
$ rustup target add x86_64-pc-windows-gnu
$ rustup toolchain install stable-x86_64-pc-windows-gnu
$ cargo build -p trin --target x86_64-pc-windows-gnu

Releases

This section covers the process of making & deploying a Trin release.

Release checklist

First

Ensure that the CI for the latest commit to master is passing. This ensures that trin itself is working, and that the latest docker image is working and published.

Communicate

Announce in #trin chat the upcoming release. Aim for a day or more notice, but announcing a few minutes before releasing is still better than not saying anything.

Choosing a version

Make sure that version follows semver rules e.g (0.2.0-alpha.3).

For the time being, ALWAYS specify the -alpha suffix.

Release the version

We use automated github release workflow to create a new release. This will create a new release draft with the new tag and will build all the binaries and attach them to the release.

1. Checkout and rebase local master to upstream

git checkout master
git pull --rebase upstream master

2. Create a new git tag with the chosen version, for example:

git tag v0.1.0-alpha.15

3. Push the tag to the upstream repository:

git push upstream v0.1.0-alpha.15

4. Wait for the github actions release job to finish. It will create automatically a draft release with all precompiled binaries included. This should take 15-20 min to complete.
5. Find the draft release generated by the github bot in releases and edit the template by completing and deleting all checklists. Write a short summary if available. Add any clarifying information that's helpful about the release.
6. Scroll to the bottom, check the Set as a pre-release box and click Publish release.

Build Instructions

• Linux
• Mac OS
• Raspberry Pi
• Windows

Deploy

Push these changes out to the nodes we run in the network. See next page for details.

Communicate

Notify in Discord chat about the new release being complete.

As trin stabilizes, more notifications will be necessary (twitter, blog post, etc). Though we probably want to do at least a small network deployment before publicizing.

Deploy trin to network

First time Setup

• Get access to cluster repo (add person to @trin-deployments)

• git clone the cluster repo: https://github.com/ethereum/cluster.git

• Install dependencies within cluster virtualenv:

  cd cluster
  python3 -m venv venv
  . venv/bin/activate
  pip install ansible
  sudo apt install ansible-core
  

  On mac you can do brew install ansible instead of apt.

• Install keybase

• Publish your pgp public key with keybase, using: keybase pgp select --import

  • This fails if you don't have a pgp key yet. If so, create one with gpg --generate-key

• Install sops

• Contact @paulj, get public pgp key into cluster repo

• Contact @paulj, get public ssh key onto cluster nodes

• Make sure your pgp key is working by running: sops portal-network/trin/ansible/inventories/dev/group_vars/secrets.sops.yml

• Log in to Docker with: docker login

• Ask Nick to be added as collaborator on Docker repo

Each Deployment

Prepare

• Generally we want to cut a new release before deployment, see previous page for instructions.
• Announce in Discord #trin that you're about to run the deployment
• Make sure to schedule plenty of time to react to deployment issues

Update Docker images

Docker images are how Ansible moves the binaries to the nodes. Update the Docker tags with:

docker pull portalnetwork/trin:latest
docker pull portalnetwork/trin:latest-bridge
docker image tag portalnetwork/trin:latest portalnetwork/trin:testnet
docker image tag portalnetwork/trin:latest-bridge portalnetwork/trin:bridge
docker push portalnetwork/trin:testnet
docker push portalnetwork/trin:bridge

This step directs Ansible to use the current master version of trin. Read about the tags to understand more.

Run ansible

• Check monitoring tools to understand network health, and compare against post-deployment, eg~
  • Glados
  • Grafana
• Activate the virtual environment in the cluster repo: . venv/bin/activate
• Make sure you've pulled the latest master branch of the deployment scripts, to include any recent changes: git pull origin master
• Go into Portal section of Ansible: cd portal-network/trin/ansible/
• Run the deployment: ansible-playbook playbook.yml --tags trin
• Run Glados deployment: updates glados + portal client (currently configured as trin, but this could change)
  • cd ../../glados/ansible
  • ansible-playbook playbook.yml --tags glados
• Wait for completion
• Launch a fresh trin node, check it against the bootnodes
• ssh into random nodes, one of each kind, to check the logs:
  • find an IP address
  • ssh ubuntu@$IP_ADDR
  • node types
    • bootnode: trin-*-1
    • bridge node: trin-*-2
    • backfill node: trin-*-3
    • regular nodes: all remaining ips
  • check logs, ignoring DEBUG: sudo docker logs trin -n 1000 | grep -v DEBUG
• Check monitoring tools to see if network health is the same or better as before deployment. Glados might lag for 10-15 minutes, so keep checking back.

Communicate

Notify in Discord chat about the network nodes being updated.

Update these docs

Immediately after a release is the best time to improve these docs:

• add a line of example code
• fix a typo
• add a warning about a common mistake
• etc.

For more about generally working with mdbook see the guide to Contribute to the book.

Celebrate

Another successful release! 🎉

FAQ

What do the Docker tags mean?

• latest: This image with trin is built on every push to master
• latest-bridge: This image with portal-bridge is built on every push to master
• testnet: This tag is used by Ansible to load trin onto the nodes we host
• bridge: This tag is used by Ansible to load portal-bridge onto the nodes we host

Note that building the Docker image on git's master takes some time. If you merge to master and immediately pull the latest Docker image, you won't be getting the build of that latest commit. You have to wait for the Docker build to complete. You should be able to see on github when the Docker build has finished.

Why can't I decrypt the SOPS file?

You might see this when running ansible, or the sops check:

Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  32F602D86B61912D7367607E6D285A1D2652C16B: FAILED
    - | could not decrypt data key with PGP key:
      | github.com/ProtonMail/go-crypto/openpgp error: Could not
      | load secring: open ~/.gnupg/secring.gpg: no such
      | file or directory; GPG binary error: exit status 2

  81550B6FE9BC474CA9FA7347E07CEA3BE5D5AB60: FAILED
    - | could not decrypt data key with PGP key:
      | github.com/ProtonMail/go-crypto/openpgp error: Could not
      | load secring: open ~/.gnupg/secring.gpg: no such
      | file or directory; GPG binary error: exit status 2

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

It means your key isn't working. Check with @paulj.

If using gpg and decryption problems persist, see this potential fix.

What do I do if Ansible says a node is unreachable?

You might see this during a deployment:

fatal: [trin-ams3-18]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host 178.128.253.26 port 22: Connection timed out", "unreachable": true}

Retry once more. If it times out again, ask @paulj to reboot the machine.

What if everything breaks and I need to rollback the deployment?

If you observe things breaking or (significantly) degraded network performance after a deployment, you might want to rollback the changes to a previously working version until the breaking change can be identified and fixed. Keep in mind that you might want to rollback just the bridge nodes, or the backfill nodes, as opposed to every node on the network.

1. Go to the commit from the previously released version tag. Click into the CI workflows for that commit and look for the docker-publish or docker-publish-bridge flow, depending on what images you want to rollback.
2. In the logs for these flows, find the sha256 digest from the Publish docker image to Docker Hub step.
3. Pull this specific image locally, using docker pull portalnetwork/trin@sha256:<HASH>
4. Retag the target image to this version, for example, if you want to re-deploy the bridges, do: docker image tag portalnetwork/trin@sha256:6dc0577a2121b711ae0e43cd387df54c8f69c8671abafb9f83df23ae750b9f14 portalnetwork/trin:bridge
5. Push the newly tagged bridge image to Docker Hub. eg. docker push portalnetwork/trin:bridge
6. Re-run the ansible script, which will use the newly updated image. Use the --limit cli flag if you only want to redeploy a subset of nodes. eg: ansible-playbook playbook.yml --tags trin --limit backfill_nodes.
7. Verify that the network is back to regular operation.

Tests

Testing is essential to the production of software with minimal flaws. The default should always be writing tests for the code you produce.

Testing also introduces overhead into our workflow. If a test suite takes a long time to run, it slows down our iteration cycle. This means finding a pragmatic balance between thorough testing, and the speed of our test suite, as well as always iterating on our testing infrastructure.

Unit test names should unambiguously identify the functionality being tested. Omit any "test" prefix from the name to avoid redundancy.

Contribute to trin Book

Using the book

The book can be built and served locally.

Installing book tools

The first time you work on the book, you need to install mdbook and the mermaid tool to generate diagrams:

cargo install mdbook mdbook-mermaid
cd book/
mdbook-mermaid install
cd ..

This will create mermaid.min.js and mermaid-init.js files.

Running the book server

Then run the book from the book crate:

cd book/
mdbook serve --open

Or, from the project root:

mdbook serve --open ./book

Adding new pages

Add a new entry to ./book/src/SUMMARY.md. Follow the style there, which follows strict formatting. There are two kinds of additions:

• New single section
  • Tab to the appropriate depth
  • Add a [Section name](section_name.md)
• New nested section
  • Tab to the appropriate depth
  • Add a [Section name](section_name/README.md)
    • Add [Subsection one](section_name/subsection_one.md)
    • Add [Subsection two](section_name/subsection_two.md)

Don't create these pages, the ./book/src/SUMMARY.md file is parsed and any missing pages are generated when mdbook serve is run. Content can then be added to the generated pages.

Then run serve:

mdbook serve --open

Test

To test the code within the book run:

mdbook test

Links

To keep the book easy to manage, avoid:

• External links likely to change
• Internal links to other pages or sections

Relative links to locations outside the ./book directory are not possible.

Diagrams

Diagrams can be added using mermaid annotations on a normal code block:

    ```mermaid
    graph TD;
        A-->B;
        A-->C;
        B-->D;
        C-->D;
    ```

The above be converted to the following during book-building:

graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;

Crate documentation location

Workspace crates are published to crates.io and include the README.md in the root of the crate. This is valuable to have when using the crates outside of the context of Trin E.g., ethportal-api.

Any documentation of workspace crates in the book should therefore be limited to explaining how the crate interacts with the other workspaces in the context of Trin. Rather than moving the workspace README.md's to the book.