Archive Node File Sharing

🧠 Project

Archive Node File Sharing

🔍 Description

Archive Node File Sharing is a lightweight peer-to-peer block retrieval system designed to reduce complexity of running an archive node! It qualifies ALREADY CONNECTED nodes in realtime, prioritizes them based on bandwidth and block height, and efficiently fetches missing blocks. This greatly reduces configuration time, planning or executing manual s3 sync commands.

🛠️ How It Works

The client tracks peers’ highest known block and available capacity, dynamically building a ranked list of capable peers. A request loop iterates through the best candidates first, retrying with backups if failures occur.

Key implementation highlights:

• Rust implementation with a tokio spawn of the async tasks inside the existing reth binary.

• Runtime peer scoring (capacity + block availability).

• Custom lightweight protocol.

• Debugging logs for real-time inspection of peer selection.

Key concepts:

• What each peer knows: Only one number — its highest block archive file (how far it has synced).

• How they discover others: When a regular peer comes online, it quickly asks it, “What’s your highest block?” It saves those answers in its own local list (no central server).

• How they decide what to get: A peer looks at its own height vs. others’. If someone is ahead, that means they have blocks we’re missing.

• How they pull data: The peer requests the next missing block from any peer that’s ahead. If one peer fails, it tries another from its list.

• What happens over time: Everyone keeps repeating this (ask heights → fetch next block). The swarm naturally catches up until all peers have the same latest block.

• Why this is reliable: No single point of failure, works even if some peers drop in/out, and it’s efficient because peers only pull blocks they know others have.

• How to tell it’s working: Your peer’s highest block number goes up steadily and eventually matches the highest number reported by others.

• Why is this better than S3: There's no centralized authority to censor us. There's no cost associated with downloads. It is low configuration and easier than following manual steps.

💻 Demo

• 👉 https://youtu.be/gKla4H-LCvw

📂 Project Links

• PR: https://github.com/hl-archive-node/nanoreth/pull/37

• Designs / Visual assets: attached diagram

👥 Team

Single-contributor: wwwehr