Blockchain

The Four Layers of the Blockchain

/ Leave a Comment

The recent DAO hack has called into question the speed and aggressiveness with which the Blockchain (and specifically Ethereum) community has moved forward on Blockchain applications. Amid many calls for reflection, one that I found especially convincing is Joi Ito’s call for a layered approach to Blockchain design (ironically posted just before the hack occurred).

While some have already proposed a layered approach to the Blockchain,many (but not all) treat the Blockchain as the base layer, which is like building a skyscraper starting at the 30th floor. Focusing on deploying complex applications on top of a Blockchain is, in my opinion, premature, as the DAO hack has demonstrated.

Instead, the Blockchain itself should be decomposed into separate layers in order to better understand the security and economics of Blockchain design. Below I propose such a decomposition and argue why it is both natural and useful.

First a recap: layered system design is best exemplified in the Internet. Each layer is more abstract than the lower layer, until we get to the physical transport layer. This approach allows for robust system design because each layer can be upgraded, patched, or even completely swapped out without affecting other layers. To give an over-simplified summary, on the Internet you have:

  • The physical layer: the actual medium that transports the bits whether it be wireless spectrum, cable fiber, or phone lines.
  • The network layer: manages addressing and routing of packets between different physical routers, most commonly IP.
  • The transport layer: manages raw connection state, most commonly TCP.
  • The session layer: manages higher level connection state, such as HTTP.
  • The application layer: where actual applications live, e.g. Google search, Facebook, etc.

This layered design means that TCP/IP functions just as well over wireless LTE as over fiber. (This point is a little trickier at the application end of the stack, but largely still holds: you could run Facebook over HTTP/2 just as well over as over HTTP/1.1.)

A Bitcoin layer decomposition

So, what might such a layered decomposition look like for Blockchain applications? Here is one proposal:

  • Consensus layer: a protocol that describes the format of a ledger that is publicly visible and a consensus function that anyone can use to determine which of multiple candidate ledgers is the consensus ledger. The protocol must also allow new blocks to be added to the ledger.
  • Mining layer: a protocol that incentivizes parties to maintain the consensus and add blocks to the ledger.
  • Propagation layer: a protocol that determines how the ledger and blocks are transmitted between nodes in the network.
  • Semantic layer: a specification of how new blocks must relate to previous blocks and a protocol for verifying conformity with the specification.
  • Application layer: application code that implements some desired functionality.

The first four layers encompass what we think of as the Blockchain, while the application layer allows for overlays, APIs, applications, etc.

Before continuing, I want to point out that the consensus layer described above is different from the one used in Eric Lombrozo’s decomposition. His use of “consensus layer” corresponds to our consensus + mining + semantic layers combined, while his “peer-to-peer layer” corresponds to our propagation layer.

Why four layers instead of one?

Treating the Blockchain as a single layer is like lumping everything between the physical and transport layers in the Internet into a single layer. While it’s fine to ignore these layers and treat them as one once they’re mature, at this moment Blockchain technology is too young and we don’t understand these base layers well enough.

By separating out the Blockchain into multiple layers, we can better study the various properties that we want the Blockchain to enjoy and where they need to be implemented. These properties are:

  • Security: no party who doesn’t control a majority of some scarce resource (typically computing power) can convince nodes that an alternate version of the ledger is the consensus
  • Liveness: nodes can add new blocks to the ledger with acceptable latency
  • Stability: nodes in the network should not alter their opinion of the consensus ledger
  • Correctness: only blocks that represent valid transactions (i.e. they conform to a specification of how new blocks may relate to previous blocks) may be added to the ledger

These properties are drawn from the survey of Bonneau et al. though we chose to rename [eventual consensus → stability] and [exponential convergence → security] for clarity’s sake.

Our classification into four layers is natural because it’s quite easy to identify that each of these properties is achieved primarily at one layer of our decomposition:

  • Security is achieved at the consensus layer, and requires building a consensus function that cannot be fooled into accepting an alternate ledger without using a majority of all existing resources
  • Liveness is achieved at the mining layer, and requires there to be enough incentive for participants in the network to continually confirm new blocks
  • Stability is achieved at the propagation layer, and requires nodes to be able to quickly disseminate confirmed blocks to other nodes so that they know to build on the most recent blocks instead of older, stale blocks
  • Correctness is achieved at the semantic layer, where blocks have a meaning, which could range from sending currency between parties as in Bitcoin to encoding state transitions in a state machine as in Ethereum, and where this meaning is validated by nodes to conform to the specification stating how new blocks must relate to previous blocks

Before moving on, I’d like to acknowledge this relationship between properties and layers is not strict and that different properties and layers are not totally independent. However, they are distinct enough to warrant study in isolation in order to gain better understanding. (Note that interdependency of layers is true of the Internet as well; HTTP runs on TCP/IP because it provides a reliable connection, but not on UDP.)

I’d also like to point that the term security is used above in a narrow sense to only mean the security of the underlying protocol. There are attacks that rely on creating unintended transactions (this is what occurred in the DAO hack) that may be called security problems, but are the result of buggy code and not a security breach in the protocol itself.

Which layer does cryptocurrency live at?

Because existing Blockchains including Bitcoin and Ethereum work at all four layers (consensus+mining+propagation+semantic) simultaneously, it’s not immediately clear at which layer cryptocurrency “lives”. In fact, it lives at two layers and in two different forms. This fact is implicit in Bitcoin and explicit in Ethereum.

  • The mining layer: Bitcoins and Ether are created and/or transferred as valid blocks are created and added to the ledger. The currency is either generated from the network itself (“out of thin air”) or taken from the transactions contained in the block (“transaction fees”). The currency is used to maintain an incentive for miners to hash blocks.
  • The semantic layer: Bitcoins and Ether can be transferred among nodes at the semantic layer by creating valid transactions signed by the holders of the cryptocurrency or by creating smart contracts that transfer the cryptocurrency between accounts. Here the cryptocurrency is used as a store of value and means of payment.
    Since Ethereum is a general-purpose VM, you can also create alternate cryptocurrencies at the semantic layer to be used in the application layer. For example, DAO tokens functioned this way. These alternate cryptocurrencies live only at the semantic and application layers.

While the fact that cryptocurrency is used to incentivize mining makes them valuable and hence attractive as an asset at the semantic layer, there is no reason that they should be the primary asset at the semantic layer. We’ll argue farther down that there is a strong case to be made for strictly limiting the mining cryptocurrency to the mining layer.

Working with the layers

This decomposition allows us to reason about aspects of the Blockchain in manageable chunks, and should help us provide more rigorous arguments for evaluating design choices when building Blockchain technologies.

In a forthcoming paper, I will present a simple theoretical proof of security of the Bitcoin Blockchain protocol, whose simplicity is possible because it only requires looking at the consensus layer. I believe that this (or a similar) decomposition into layers will open the door to simple analyses of the higher layers as well.

A first proposal: restriction on mining coins

Let me end the post with a proposal based on this layering that could help improve the security and flexibility of Blockchains.

As mentioned above, existing cryptocurrencies live both at the mining layer and the semantic layer of the Blockchain stack.

We argue that mining coins (the cryptocurrency used at the mining layer) should be usable only at the mining layer, and even more stringently they should only support two operations: creation and consumption. In addition, consuming a mining coin doesn’t mean transferring it from the publisher to the miner, consuming a mining coin means attaching it to a block andretiring the coin when that block is confirmed, sending it back into the void where it came from.

Note that we have explicitly prohibited mining coins from being transferred between nodes. This restriction improves security because it means that it’s literally impossible to steal mining coins, eliminating fraud at the mining layer!

Of course, the question becomes how do people get their hands on enough mining coins to use the Blockchain? The distribution of mining capacity is uneven and will be concentrated in the hands of dedicated miners, so we must have some means of reallocating mining coins.

This is solved by user coins. User coins live at the semantic level and can be used as a store of value and means of payment, meaning in particular that they are transferrable. (Just to clarify, calling them “user coins” does not mean each user gets their own kind of coin. There may be one type of user coin for the entire system, or there could be more than one, it depends on that particular Blockchain’s design. The term “user coin” is meant to emphasize that these are the coins that should be used by end-users to handle non-mining related transactions.)

A node who wishes to publish a transaction t but doesn’t have any mining coins to spend on publishing will attach a related contract t’ that says “transfer n user coins to the node who attaches enough mining coins to allow both t and t’ to be published”. A miner who has mining coins could then take t and t’, contribute enough mining coins to the block, and publish them together in order to obtain the n user coins.

Separate currencies for mining and semantic layer

This might seem a roundabout way to achieve something that can already be done with a single type of coin, but in fact a strict separation of user and mining coins has several advantages:

  • User coins can be structured using a different economic model than mining coins. For example the creation and transfer of user coins can have rules that reflect a central banking model, a community currency model, etc. Multiple models can exist on the same Blockchain. The user coins can also have fraud prevention measures, e.g. a trusted third party who is allowed to reverse certain transactions.
  • Even if one type of user coin is compromised, the system can survive by switching to a different user coin. Thus, the fundamental viability of the network can be decoupled from the economics of the semantic layer (at least to some extent).
  • The issuance models of existing cryptocurrencies are extremely ad hoc: Bitcoin has a predefined schedule of Bitcoin creation, Ethereum has a somewhat different schedule, but none of them use a model where the money supply adapts to economic conditions, which is something that is useful for moderating the effects of business cycles. Typical arguments for using these arbitrary models are that prices / interest rates will adjust to take into account the issuance rate, but this is a cop-out. In practice price levels are important psychological signposts when making economic decisions, and ideally a currency system will have some mechanism to promote price stability (not necessarily zero inflation, but hopefully low and predictable inflation).
  • This dual mining / user coin system allows for flexible issuance models. First, mining coins, are transient by nature and their supply adapts naturally to the rate of block creation: they are created when blocks are confirmed and are also retired when blocks are confirmed, which means that demand and supply of mining coins should move together and should lead to a stable price for mining coins.
    Second, user coins can use any issuance model, and so decoupling the mining and user coins enables less risky experimentation with user coin issuance models.

Of course this is a cursory analysis and pinning down the exact issuance and consumption model of mining coins is an important topic for further research and I’ll share some more detailed ideas in future posts.

One wishful and one practical suggestion for mitigating DAO-style hacks

/ Leave a Comment

Excellent description of the nitty-gritty of the DAO hack over here on Peter Vessenes’ blog. I wanted to jump on this opportunity to point out that another reason this hack was possible was because the creators of the DAO contract were using the wrong paradigm for thinking about money. We shouldn’t blame them for this particular mistake (they have much more grave mistakes to answer for) because it’s the paradigm we all use, but that doesn’t make it any less wrong, and understanding the problem may help guide future development.

When you think of money, what’s the first thing that comes to mind? For anyone born before the 21st century, it’s probably coins and bills, which is the first kind of money that most people are exposed to. The fact that these are physical tokens colors the way we think about money from our childhood onwards, and leads to the belief that money is the fundamental unit of finance. But this is not true!

The fundamental unit of finance is not money, but the exchange of money.

Our human brains are much more adept at thinking about nouns than about verbs, so we’ve focused our attention on the technology that records the exchange of money rather than the underlying exchange itself.

But think about it, coins and bills (and to go back further, gold and seashells) are no more than a technology that helps us keep track of the past history of economic transactions. Their primary advantage is that they’re not easily counterfeited, so when someone hands us a wad of cash we believe that there was a history of economic transactions that led that person to earn that wad of cash.

Thinking about it this way, cash and bills are a rather poor technology in our digital age. Paper bank ledgers are better, digital bank ledgers are much better, and decentralized ledgers are (might be?) even better still, because they contain much richer information about the history of exchanges. Indeed Blockchain ledgers are organized this way: there is a public record of all transactions and account balances can be deduced form them. (The fact that Ethereum explicitly tracks balances of accounts is in fact unnecessary and possibly useful only for optimization reasons.)

Returning to the incorrect paradigm, as humans with our proclivity to focus on nouns, we think much more readily about account balances, the digital analogue of tokens, than we think about histories of transactions. But an account balance is nothing more than a summary of a history of transactions, and so an account balance is just the output of running a function on a history of transactions.

This function seems simple: subtract outgoing transfers and add incoming ones. If we had a ledger that didn’t change this would be fine. But this ignores the dynamic nature of ledgers, where new transactions are constantly being added and those transactions require some integrity guarantee (e.g. no new transaction should be added that makes the net total transfer go negative) and violating transactions should be rejected.

How did this lead to the DAO hack?

I’ll again defer to this excellent recap of the hack for the details, but I’ll try to summarize the hack using the above perspective on account balances vs. transaction histories. The DAO contract checks whether certain transactions are legal by comparing them to a stored record of account balances (red flag!). If the transaction does not cause the account balance to go negative, then the transaction is permitted. After the transaction is created the account balance is updated to reflect the outgoing transfer.

The hacker was able to exploit two bugs where the transaction history and account balance fell out of sync. The first bug is subtle: the code allowed for a recursive call where two or more transactions are recursively created. Both transactions attempt to move out the attacker’s entire balance out of the DAO. When it succeeds, the inner transaction deducts the attacker’s previous balance from the DAO’s balance and sets the attacker’s remaining funds balance to zero. The outer transaction also completes, and deducts the attacker’s previous balance from the DAO’s balance. But the attacker’s previous balance was just updated to zero by the inner transaction! So after the outer transaction completes the DAO has not only transferred twice the amount it should have to the attacker, it also thinks it has more funds than it actually does.

The second bug is a glaring error: before the withdrawal happens there is a function call that looks like it’s supposed to reduce the attacker’s balance even before the transfer is created. This call is unfortunately mistyped to call a simple logging function instead. Had it been called correctly, the attack would have been prevented. (Although, looking closely at the code, calling the function that reduces the attacker’s code balance would have caused a different though less severe bug. It may have been a sloppy fix to this less severe bug that introduced the vulnerability.)

Suggestion 1: stop using account balances

The first and most radical suggestion is that developers stop using account balances in their code. Instead, checking whether or not a transaction is valid (e.g. there is enough Ether to transfer out) should always be done by looking at the history of transactions and recomputing the net total transfer into/out of an account.

This is admittedly computationally expensive, and in a resource-constrained environment like the EVM may be a non-starter. Nevertheless perhaps this philosophy can help guide future development of the EVM or successor technologies.

Suggestion 2: create SQL-like commit/rollback functionality

This error would also have been prevented if transaction creation and account balance update were atomic: both succeed or fail together and at the same time. This is a basic standard notion in applications like databases and surprisingly absent from the Ethereum spec, given the importance of data integrity in financial settings.

In this particular case, wrapping everything in a transaction would have resulted in a final commit that tried to create multiple transactions and deduct the total amount of all the requested transactions from the account balance, which would have been rejected since the account did not have a sufficient balance.

I see no reason why the EVM couldn’t be modified to support such functionality. Maybe it’s even possible to support this functionality at a higher level without modifying the EVM spec. The protection it provides is orthogonal to some of the other tools that people are proposing (strong type safety, formal verification, etc.) so they should all be pursued, but I would guess that this is both simpler and more user-friendly since most people with some dev experience have used a database commit/rollback before.

Update: Thanks to edmundedgar on the reddit channel for helping me see that the above suggestion about commit/rollback isn’t really appropriate, since the EVM execution environment is single-threaded. The problem is not contesting threads, its re-entrant calls on a single thread.

I think a related solution would still help though: create a standard library for managing transfers of balances (could be both for tokens or Ether). An example transfer function might do the following: keep track of a “transfer active” flag per account, and when a transfer into/out of an account is attempted, first check the flag. If it’s on then throw an error, otherwise perform the transfer, update the account balance, and clear the flag. In this case, if the receiving contract attempts a re-entrant call, the flag will be on and an error will be thrown, thwarting the attack.

Final notes on terminology

I’ve avoided using the word “database transaction” (which is what this kind of commit/rollback mechanism is doing) since the word transaction is overloaded here. Any suggestions on how to refer to this idea without using the word transaction?

I also think it’d be nice if we had a simple and precise word to capture the notion of “exchange of money”. I don’t have the best words, but someone who has better words may be able to come up with a fitting and memorable term that we can use to help put the exchange of money at the center of our thinking about financial transactions rather than the money itself.