- In the absence of any centralised authority, in decentralised systems there needs to be a way for everyone to come to an agreement.
- A consensus mechanism is the way in which different stakeholders in the blockchain ecosystem agree on a given state of the blockchain. Agreeing on its current state is a prerequisite for the functioning of a blockchain.
- There are several ways to achieve a consensus, but there cannot be a “one-size-fits-all” approach. Consensus mechanisms have different trade-offs, such as speed, security and decentralisation. As blockchains are built for a specific purpose, the consensus mechanism needs to be selected accordingly.
The necessity of consensus mechanisms in decentralised systems
With the evolution of commerce, emails have replaced letter mail, databases have replaced physical ledgers, and so on. However, trust, the most fundamental factor in any transaction, remains the same. For two people to engage in a transactional contract, trust is essential – either in each other or in a third party possessing the power to enforce the contract. Public blockchains1 such as Bitcoin have minimised the trust that needs to be placed in the other party by maintaining a decentralised, single version of the truth. And to do this, they need consensus mechanisms.
Consensus mechanisms are designed so that the incentives of the network and individuals are aligned to the maximum possible extent. In other words, each individual stakeholder, though selfish, is given an incentive to behave in a way that is good for the whole network.
We will now explore the Byzantine General’s problem in order to gain a better understanding of the need for a consensus mechanism in decentralised systems.
The Byzantine General’s problem
This problem troubled computer scientists for a long time and remained unsolved until the Bitcoin protocol was released. According to the problem, the Eastern Roman Empire, or Byzantine Empire, has decided to capture a city that is offering fierce resistance. The three divisions of the Byzantine army, each led by a General, have encircled the city. Each division has two options: attack or retreat. Victory can be achieved only if all the Generals agree on a particular strategy, otherwise they will suffer a brutal defeat. The Generals must communicate with each other to ensure that everyone is in agreement. They can only communicate through messengers, and can only send their message once.
However, there are several potential problems. One or more Generals could be traitors. The messengers could be delayed, killed or compromised. With all these constraints, in a physical world it would be almost impossible for the Generals to reach a consensus and take the city. If you were one of the Generals, how would you act on any message if you didn’t know whether the person who had sent it to you was trustworthy and whether the message had been intercepted?
In the above scenario where General 2 is a traitor, there is no way for General 3 to know whether to attack or retreat as he receives contradictory messages from the other Generals. Thus, a consensus cannot be reached, and the attack will not be successful (Exhibit 1).
Exhibit 1: Difficulty in achieving a consensus when there is a traitor
Each General can be regarded as one of the nodes on the network. As the number of nodes increases, achieving a consensus becomes more complicated and less reliable. The solution to the problem lies in ensuring that all the Generals can agree on a strategy without relying on anything else. There are two critical elements within such a design:
- Encrypting the messages
- Giving network participants an incentive: punishing cheating and/or rewarding good behaviour
Encryption using hash functions
The messages sent by the Generals need to be encrypted using hash functions to make them tamperproof. Hash functions are one-way functions that encrypt a message so that it becomes virtually impossible to obtain the original message from the output. We invite readers to check for themselves using the following link.
SHA-256, for Secure Hash Algorithm variant 256, is used in the Bitcoin protocol. Regardless of the input text, the output is always a string of 64 digits. Minor changes in the original message cause massive differences in outputs (Exhibit 2). The hash of “attack” is completely different from the hash of “Attack”, even though the change in the original message appears to be trivial.
Exhibit 2: SHA-256 output
The use of hash functions gives the Byzantine Generals a higher degree of confidence in the messages they receive. To ensure that a message has not been tampered with, Generals add a random number along with the message. This random number is called nonce. Adding nonce provides extra security, as anyone who wishes to change the message must also change the nonce for the output to remain acceptable2. Only the Generals know what the acceptable output is. Therefore even minor tampering with the message will change the output and the recipient will know that there has been an attempt to change the message.
The role of incentives in consensus mechanisms
Using encryption alone does not fully solve the problem. A traitor General can still find a nonce that makes the output acceptable with a “retreat” message instead of “attack”. So how can it be ensured that the General is acting in the best interests of the Empire? This is done by making misbehaviour costly for traitors. Different consensus algorithms have different methods of aligning incentives for network participants. Without this alignment, there is room for misbehaviour.
We will now present the two most widely used consensus mechanisms.
Proof of work (PoW)
What is PoW?
Satoshi Nakamoto developed PoW in order to achieve a consensus on the Bitcoin network. The consensus process can be broken down into two steps: proposing the current network state – the single truth – and accepting the current network state. Proof of work makes it expensive to propose the current state of the network. In Bitcoin’s consensus mechanism, the miners are the ones who propose the existing state of network, along with the right nonce to enforce trust in the system. Finding the right nonce is very unusual and can only be done by brute force. Therefore, discovering the correct nonce is expensive. However, checking the nonce is very easy. Miners are paid if their proposition is accepted by the network. As it is easy to check the combination between the nonce and the current network state proposed by the miners, there is little incentive for them to submit the wrong state. Consequently, it is in the best interests of the miners to submit the correct state every time: the single truth.
Incentive alignment in PoW using the Byzantine Empire analogy
If the Byzantine Empire were to employ PoW, they would have to promise the spoils from the city to the Generals if the attack succeeded. If the attack failed, the Generals would not receive anything. To compel the Generals to act in the best interests of the Empire, the following additions must be made to the design;
- Generals need to invest significant capital to set up the messaging system
- The generation of each message must be expensive
Generals need rewards to recover the costs they have incurred to set up the messaging system. As they are rewarded only when the attack is successful, they are compelled to work towards making the attack successful.
- Most secure consensus algorithm
- Miners cannot hoard coins as they need to sell them to cover their operational costs
- Inefficient in terms of energy – brute force is used
- Longer transaction time – the right nonce is only found after trial and error
- High fixed and operational costs
Examples of crypto-assets using PoW: Bitcoin, Ethereum (plans to migrate to proof of stake), Ethereum Classic, Monero, Zcash.
Proof of stake (PoS)
What is PoS?
In a proof of stake consensus, validators or forgers have the same functions as the miners in PoW. The mechanism is different, however. To participate in the consensus, validators have to stake a minimum amount in the blockchain’s native token to be selected to validate a block and earn transaction fees.
Incentive alignment in PoS using the Byzantine analogy
If the Byzantine Empire were to implement PoS, there would have to be a condition according to which anyone aspiring to be a General would have a significant stake in the Empire. If they do not act in the best interests of the Empire, any loss in the Empire’s value would also hurt their possessions. The best strategy for the Generals is therefore to act in the interests of the Empire.
Unlike PoW, with a PoS consensus there is no race among the users to find the next block. A user needs to own a stake in the network to be regarded as a validator, which forces users to put as much effort as possible into the game. Validators are chosen randomly. Two common ways to select a block forger are the Coin Age Selection3 and the Randomised Block Selection4 processes.
- Efficient in terms of energy
- Shorter transaction time
- No need to buy expensive mining equipment
- As a validator is selected before the block has been proposed, transaction manipulation is possible through bribing
- The lack of operational costs represents an incentive for re-staking, which may lead to hoarding
- “Nothing at stakeNothing at stakelink1 problem5”. Simply having nothing at stake means that there is no variable cost associated with voting on the legitimacy of the block, so it is possible for validators to vote on multiple blocks at a time and receive rewards for the one that is accepted by the network.
Example of crypto-assets using PoS: Algorand, Cosmos, Binance Coin, Qtum.
The need to place trust in others has long been the status quo of the financial world. In a decentralised ecosystem, the consensus mechanism plays an important role in resolving the trust issue to a large extent. By aligning an individual’s incentives to those of the network, consensus mechanisms enable a single truth to emerge, creating trust in the system.
However, no consensus mechanism is perfect as there must be trade-offs between elements such as speed, decentralisation and security. The consensus to be employed depends on the design goal of the blockchain. Understanding the consensus mechanism of a blockchain forms the bedrock for discerning what that blockchain can and cannot do.
1Public blockchains are open to everyone, while private blockchains are by invitation. ↵
2Acceptable output defines certain standards for the output. For example, in the case of Bitcoin, it represents a certain number of 0s at the beginning of the hash value. ↵
3A validator is selected based on the age of the coins staked ↵
4A forger is selected based on the combination of various parameters such as stakes, age, recentness etc. ↵
5A detailed explanation of the “nothing at stake problem” is beyond the scope of this document. ↵