...by Daniel Szego
"On a long enough timeline we will all become Satoshi Nakamoto.."
Daniel Szego

Saturday, August 11, 2018

How to implement a Blockchain from scratch - network protocols

At creating a blockchain solution from scratch, one of the most important part is to design a set of network protocols. It is important to note that these protocols are on the application level, under the hood they might be realized by simple socket sending on a TCP, something more abstract RPC or JSON-RPC, or even with onion routing. At any case, the following protocols should be considered:

0. Get client version: connecting to a known peer and getting the version of that peer. It prevents the usage of incompatible peers, that might be highly important especially if there are updates of the code regularly.

1. Update peer information: A brand new peers is usually started with a couple of preconfigured nodes further peer information. These peer information are queried for the further known peers until the number of known peers reach a certain limit (like 15 in Bitcoin by default). The peers may go offline and come back online again, therefore the active peers have to be checked regularly if they are still alive, if not different strategies can be implemented, like:
- deleting the inactive peer from the cached peer list.
- waiting for a certain time if the peer will be online again.
- querying all the still active node to get more peers that are hopefully active.
- or a combination of the previous strategies.

2. Syncronizing the blocks and states: As a first step of using the blockchin, the blockchain must be more or less up to date with the rest of the world. As a first step, the node can connect with all of the peers and ask the size of the blockchain. Based on that information, it knows exactly if the blockchain needs to be synchronized or not. As a second step a

3. Propagating transactions and propagating blocks: if new transactions or blocks arrive, they must be registered locally and further communicated possibly as fast as possible to all of the connecting peers in the form of a gossip protocol. If it is a transaction, it must be added to the local transaction pool, if it is a block it must be added to the possible top headers of the blockchain.  It is especially critical with the new blocks as the winner of the mining competition depends on the speed of the propagation. This phase should be available just after the blockhcain has already reached at least a quasi synchronized status. It is however an interesting general question how the propagating mechanism might stop, without effectively over flooding the network. One way might be to implement something as a finite hop system, in which a given information is propagated only at a given time. Another idea might be to implement regularly handshakes where peers exchange information on relevant transactions and blocks first before transporting them. 

In all of the communication protocols, it is always questionable, if we consider a kind of a push or pull protocol. In both protocols, the design direction should be to transfer as less amount of data as possible, and only if it is necessary. As a consequence, most protocols should transfer the inventory first, meaning the hash values of blocks and transactions. After that the nodes should be able to download the content of the hash values on-demand.