Networking layer
The Electroneum Smart Chain is a peer-to-peer network with thousands of nodes that must be able to communicate with one another using standardised protocols. The "networking layer" is the stack of protocols that allow those nodes to find each other and exchange information. This includes "gossiping" information (one-to-many communication) over the network as well as swapping requests and responses between specific nodes (one-to-one communication). Each node must adhere to specific networking rules to ensure they are sending and receiving the correct information.
Prerequisites
Some knowledge of Electroneum nodes and clients will be helpful for understanding this page.
Electroneum Smart Chain
The Electroneum Smart Chain's networking protocols is divided into two stacks:
the discovery stack: built on top of UDP and allows a new node to find peers to connect to
the DevP2P stack: sits on top of TCP and enables nodes to exchange information
Both stacks work in parallel. The discovery stack feeds new network participants into the network, and the DevP2P stack enables their interactions.
Discovery
Discovery is the process of finding other nodes in network. This is bootstrapped using a small set of bootnodes (nodes whose addresses are hardcoded into the client so they can be found immediately and connect the client to peers). These bootnodes only exist to introduce a new node to a set of peers - this is their sole purpose, they do not participate in normal client tasks like syncing the chain, and they are only used the very first time a client is spun up.
The protocol used for the node-bootnode interactions is a modified form of Kademlia which uses a distributed hash table to share lists of nodes. Each node has a version of this table containing the information required to connect to its closest peers. This 'closeness' is not geographical - distance is defined by the similarity of the node's ID. Each node's table is regularly refreshed as a security feature. For example, in the Discv5, discovery protocol nodes are also able to send 'ads' that display the subprotocols that the client supports, allowing peers to negotiate about the protocols they can both use to communicate over.
Discovery starts with a game of PING-PONG. A successful PING-PONG "bonds" the new node to a bootnode. The initial message that alerts a bootnode to the existence of a new node entering the network is a PING. This PING includes hashed information about the new node, the bootnode and an expiry time-stamp. The bootnode receives the PING and returns a PONG containing the PING hash. If the PING and PONG hashes match then the connection between the new node and bootnode is verified and they are said to have "bonded".
Once bonded, the new node can send a FIND-NEIGHBOURS request to the bootnode. The data returned by the bootnode includes a list of peers that the new node can connect to. If the nodes are not bonded, the FIND-NEIGHBOURS request will fail, so the new node will not be able to enter the network.
Once the new node receives a list of neighbours from the bootnode, it begins a PING-PONG exchange with each of them. Successful PING-PONGs bond the new node with its neighbours, enabling message exchange.
ENR: Electroneum Node Records
The Electroneum Node Record is an object that contains three basic elements: a signature (hash of record contents made according to some agreed identity scheme), a sequence number that tracks changes to the record, and an arbitrary list of key:value pairs. This is a future-proof format that allows easier exchange of identifying information between new peers and is the preferred network address format for Electroneum Smart Chain nodes.
Why is discovery built on UDP?
UDP does not support any error checking, resending of failed packets, or dynamically opening and closing connections - instead it just fires a continuous stream of information at a target, regardless of whether it is successfully received. This minimal functionality also translates into minimal overhead, making this kind of connection very fast. For discovery, where a node simply wants to make its presence known in order to then establish a formal connection with a peer, UDP is sufficient. However, for the rest of the networking stack, UDP is not fit for purpose. The informational exchange between nodes is quite complex and therefore needs a more fully featured protocol that can support resending, error checking etc. The additional overhead associated with TCP is worth the additional functionality. Therefore, the majority of the P2P stack operates over TCP.
DevP2P
DevP2P is itself a whole stack of protocols that the Electroneum Smart Chain implements to establish and maintain the peer-to-peer network. After new nodes enter the network, their interactions are governed by protocols in the DevP2P stack. These all sit on top of TCP and include the RLPx transport protocol, wire protocol and several sub-protocols. RLPx is the protocol governing initiating, authenticating and maintaining sessions between nodes. RLPx encodes messages using RLP (Recursive Length Prefix) which is a very space-efficient method of encoding data into a minimal structure for sending between nodes.
A RLPx session between two nodes begins with an initial cryptographic handshake. This involves the node sending an auth message which is then verified by the peer. On successful verification, the peer generates an auth-acknowledgement message to return to the initiator node. This is a key-exchange process that enables the nodes to communicate privately and securely. A successful cryptographic handshake then triggers both nodes to to send a "hello" message to one another "on the wire". The wire protocol is initiated by a successful exchange of hello messages.
The hello messages contain:
protocol version
client ID
port
node ID
list of supported sub-protocols
This is the information required for a successful interaction because it defines what capabilities are shared between both nodes and configures the communication. There is a process of sub-protocol negotiation where the lists of sub-protocols supported by each node are compared and those that are common to both nodes can be used in the session.
Along with the hello messages, the wire protocol can also send a "disconnect" message that gives warning to a peer that the connection will be closed. The wire protocol also includes PING and PONG messages that are sent periodically to keep a session open. The RLPx and wire protocol exchanges therefore establish the foundations of communication between the nodes, providing the scaffolding for useful information to be exchanged according to a specific sub-protocol.
Sub-protocols
Wire protocol
Once peers are connected and an RLPx session has been started, the wire protocol defines how peers communicate. There are three main tasks defined by the wire protocol: chain synchronisation, block propagation and transaction exchange. Chain synchronisation is the process of validating blocks near head of the chain, checking their data and re-executing their transactions to ensure their root hashes are correct, then cascading back in history via those blocks' parents, grandparents, etc. until the whole chain has been downloaded and validated. State sync is a faster alternative that only validates block headers. Block propagation is the process of sending and receiving newly mined blocks. Transaction exchange refers to exchanging pending transactions between nodes so that miners can select some of them for inclusion in the next block. Detailed information about these tasks are available here. Clients that support these sub-protocols expose them via the json-rpc.
les (light electroneum subprotocol)
This is a minimal protocol for syncing light clients. Traditionally, this protocol has rarely been used because full nodes are required to serve data to light clients without being incentivised. The default behaviour of execution clients is not to serve light client data over les. More information is available in the les spec.
Snap
The snap protocol is an optional extension that allows peers to exchange snapshots of recent states, allowing peers to verify account and storage data without having to download intermediate Merkle trie nodes.
Last updated