The Four Layers of the Blockchain

/ Leave a Comment
Share

The recent DAO hack has called into question the speed and aggressiveness with which the Blockchain (and specifically Ethereum) community has moved forward on Blockchain applications. Amid many calls for reflection, one that I found especially convincing is Joi Ito’s call for a layered approach to Blockchain design (ironically posted just before the hack occurred).

While some have already proposed a layered approach to the Blockchain,many (but not all) treat the Blockchain as the base layer, which is like building a skyscraper starting at the 30th floor. Focusing on deploying complex applications on top of a Blockchain is, in my opinion, premature, as the DAO hack has demonstrated.

Instead, the Blockchain itself should be decomposed into separate layers in order to better understand the security and economics of Blockchain design. Below I propose such a decomposition and argue why it is both natural and useful.

First a recap: layered system design is best exemplified in the Internet. Each layer is more abstract than the lower layer, until we get to the physical transport layer. This approach allows for robust system design because each layer can be upgraded, patched, or even completely swapped out without affecting other layers. To give an over-simplified summary, on the Internet you have:

  • The physical layer: the actual medium that transports the bits whether it be wireless spectrum, cable fiber, or phone lines.
  • The network layer: manages addressing and routing of packets between different physical routers, most commonly IP.
  • The transport layer: manages raw connection state, most commonly TCP.
  • The session layer: manages higher level connection state, such as HTTP.
  • The application layer: where actual applications live, e.g. Google search, Facebook, etc.

This layered design means that TCP/IP functions just as well over wireless LTE as over fiber. (This point is a little trickier at the application end of the stack, but largely still holds: you could run Facebook over HTTP/2 just as well over as over HTTP/1.1.)

A Bitcoin layer decomposition

So, what might such a layered decomposition look like for Blockchain applications? Here is one proposal:

  • Consensus layer: a protocol that describes the format of a ledger that is publicly visible and a consensus function that anyone can use to determine which of multiple candidate ledgers is the consensus ledger. The protocol must also allow new blocks to be added to the ledger.
  • Mining layer: a protocol that incentivizes parties to maintain the consensus and add blocks to the ledger.
  • Propagation layer: a protocol that determines how the ledger and blocks are transmitted between nodes in the network.
  • Semantic layer: a specification of how new blocks must relate to previous blocks and a protocol for verifying conformity with the specification.
  • Application layer: application code that implements some desired functionality.

The first four layers encompass what we think of as the Blockchain, while the application layer allows for overlays, APIs, applications, etc.

Before continuing, I want to point out that the consensus layer described above is different from the one used in Eric Lombrozo’s decomposition. His use of “consensus layer” corresponds to our consensus + mining + semantic layers combined, while his “peer-to-peer layer” corresponds to our propagation layer.

Why four layers instead of one?

Treating the Blockchain as a single layer is like lumping everything between the physical and transport layers in the Internet into a single layer. While it’s fine to ignore these layers and treat them as one once they’re mature, at this moment Blockchain technology is too young and we don’t understand these base layers well enough.

By separating out the Blockchain into multiple layers, we can better study the various properties that we want the Blockchain to enjoy and where they need to be implemented. These properties are:

  • Security: no party who doesn’t control a majority of some scarce resource (typically computing power) can convince nodes that an alternate version of the ledger is the consensus
  • Liveness: nodes can add new blocks to the ledger with acceptable latency
  • Stability: nodes in the network should not alter their opinion of the consensus ledger
  • Correctness: only blocks that represent valid transactions (i.e. they conform to a specification of how new blocks may relate to previous blocks) may be added to the ledger

These properties are drawn from the survey of Bonneau et al. though we chose to rename [eventual consensus → stability] and [exponential convergence → security] for clarity’s sake.

Our classification into four layers is natural because it’s quite easy to identify that each of these properties is achieved primarily at one layer of our decomposition:

  • Security is achieved at the consensus layer, and requires building a consensus function that cannot be fooled into accepting an alternate ledger without using a majority of all existing resources
  • Liveness is achieved at the mining layer, and requires there to be enough incentive for participants in the network to continually confirm new blocks
  • Stability is achieved at the propagation layer, and requires nodes to be able to quickly disseminate confirmed blocks to other nodes so that they know to build on the most recent blocks instead of older, stale blocks
  • Correctness is achieved at the semantic layer, where blocks have a meaning, which could range from sending currency between parties as in Bitcoin to encoding state transitions in a state machine as in Ethereum, and where this meaning is validated by nodes to conform to the specification stating how new blocks must relate to previous blocks

Before moving on, I’d like to acknowledge this relationship between properties and layers is not strict and that different properties and layers are not totally independent. However, they are distinct enough to warrant study in isolation in order to gain better understanding. (Note that interdependency of layers is true of the Internet as well; HTTP runs on TCP/IP because it provides a reliable connection, but not on UDP.)

I’d also like to point that the term security is used above in a narrow sense to only mean the security of the underlying protocol. There are attacks that rely on creating unintended transactions (this is what occurred in the DAO hack) that may be called security problems, but are the result of buggy code and not a security breach in the protocol itself.

Which layer does cryptocurrency live at?

Because existing Blockchains including Bitcoin and Ethereum work at all four layers (consensus+mining+propagation+semantic) simultaneously, it’s not immediately clear at which layer cryptocurrency “lives”. In fact, it lives at two layers and in two different forms. This fact is implicit in Bitcoin and explicit in Ethereum.

  • The mining layer: Bitcoins and Ether are created and/or transferred as valid blocks are created and added to the ledger. The currency is either generated from the network itself (“out of thin air”) or taken from the transactions contained in the block (“transaction fees”). The currency is used to maintain an incentive for miners to hash blocks.
  • The semantic layer: Bitcoins and Ether can be transferred among nodes at the semantic layer by creating valid transactions signed by the holders of the cryptocurrency or by creating smart contracts that transfer the cryptocurrency between accounts. Here the cryptocurrency is used as a store of value and means of payment.
    Since Ethereum is a general-purpose VM, you can also create alternate cryptocurrencies at the semantic layer to be used in the application layer. For example, DAO tokens functioned this way. These alternate cryptocurrencies live only at the semantic and application layers.

While the fact that cryptocurrency is used to incentivize mining makes them valuable and hence attractive as an asset at the semantic layer, there is no reason that they should be the primary asset at the semantic layer. We’ll argue farther down that there is a strong case to be made for strictly limiting the mining cryptocurrency to the mining layer.

Working with the layers

This decomposition allows us to reason about aspects of the Blockchain in manageable chunks, and should help us provide more rigorous arguments for evaluating design choices when building Blockchain technologies.

In a forthcoming paper, I will present a simple theoretical proof of security of the Bitcoin Blockchain protocol, whose simplicity is possible because it only requires looking at the consensus layer. I believe that this (or a similar) decomposition into layers will open the door to simple analyses of the higher layers as well.

A first proposal: restriction on mining coins

Let me end the post with a proposal based on this layering that could help improve the security and flexibility of Blockchains.

As mentioned above, existing cryptocurrencies live both at the mining layer and the semantic layer of the Blockchain stack.

We argue that mining coins (the cryptocurrency used at the mining layer) should be usable only at the mining layer, and even more stringently they should only support two operations: creation and consumption. In addition, consuming a mining coin doesn’t mean transferring it from the publisher to the miner, consuming a mining coin means attaching it to a block andretiring the coin when that block is confirmed, sending it back into the void where it came from.

Note that we have explicitly prohibited mining coins from being transferred between nodes. This restriction improves security because it means that it’s literally impossible to steal mining coins, eliminating fraud at the mining layer!

Of course, the question becomes how do people get their hands on enough mining coins to use the Blockchain? The distribution of mining capacity is uneven and will be concentrated in the hands of dedicated miners, so we must have some means of reallocating mining coins.

This is solved by user coins. User coins live at the semantic level and can be used as a store of value and means of payment, meaning in particular that they are transferrable. (Just to clarify, calling them “user coins” does not mean each user gets their own kind of coin. There may be one type of user coin for the entire system, or there could be more than one, it depends on that particular Blockchain’s design. The term “user coin” is meant to emphasize that these are the coins that should be used by end-users to handle non-mining related transactions.)

A node who wishes to publish a transaction t but doesn’t have any mining coins to spend on publishing will attach a related contract t’ that says “transfer n user coins to the node who attaches enough mining coins to allow both t and t’ to be published”. A miner who has mining coins could then take t and t’, contribute enough mining coins to the block, and publish them together in order to obtain the n user coins.

Separate currencies for mining and semantic layer

This might seem a roundabout way to achieve something that can already be done with a single type of coin, but in fact a strict separation of user and mining coins has several advantages:

  • User coins can be structured using a different economic model than mining coins. For example the creation and transfer of user coins can have rules that reflect a central banking model, a community currency model, etc. Multiple models can exist on the same Blockchain. The user coins can also have fraud prevention measures, e.g. a trusted third party who is allowed to reverse certain transactions.
  • Even if one type of user coin is compromised, the system can survive by switching to a different user coin. Thus, the fundamental viability of the network can be decoupled from the economics of the semantic layer (at least to some extent).
  • The issuance models of existing cryptocurrencies are extremely ad hoc: Bitcoin has a predefined schedule of Bitcoin creation, Ethereum has a somewhat different schedule, but none of them use a model where the money supply adapts to economic conditions, which is something that is useful for moderating the effects of business cycles. Typical arguments for using these arbitrary models are that prices / interest rates will adjust to take into account the issuance rate, but this is a cop-out. In practice price levels are important psychological signposts when making economic decisions, and ideally a currency system will have some mechanism to promote price stability (not necessarily zero inflation, but hopefully low and predictable inflation).
  • This dual mining / user coin system allows for flexible issuance models. First, mining coins, are transient by nature and their supply adapts naturally to the rate of block creation: they are created when blocks are confirmed and are also retired when blocks are confirmed, which means that demand and supply of mining coins should move together and should lead to a stable price for mining coins.
    Second, user coins can use any issuance model, and so decoupling the mining and user coins enables less risky experimentation with user coin issuance models.

Of course this is a cursory analysis and pinning down the exact issuance and consumption model of mining coins is an important topic for further research and I’ll share some more detailed ideas in future posts.


Share

One wishful and one practical suggestion for mitigating DAO-style hacks

/ Leave a Comment
Share

Excellent description of the nitty-gritty of the DAO hack over here on Peter Vessenes’ blog. I wanted to jump on this opportunity to point out that another reason this hack was possible was because the creators of the DAO contract were using the wrong paradigm for thinking about money. We shouldn’t blame them for this particular mistake (they have much more grave mistakes to answer for) because it’s the paradigm we all use, but that doesn’t make it any less wrong, and understanding the problem may help guide future development.

When you think of money, what’s the first thing that comes to mind? For anyone born before the 21st century, it’s probably coins and bills, which is the first kind of money that most people are exposed to. The fact that these are physical tokens colors the way we think about money from our childhood onwards, and leads to the belief that money is the fundamental unit of finance. But this is not true!

The fundamental unit of finance is not money, but the exchange of money.

Our human brains are much more adept at thinking about nouns than about verbs, so we’ve focused our attention on the technology that records the exchange of money rather than the underlying exchange itself.

But think about it, coins and bills (and to go back further, gold and seashells) are no more than a technology that helps us keep track of the past history of economic transactions. Their primary advantage is that they’re not easily counterfeited, so when someone hands us a wad of cash we believe that there was a history of economic transactions that led that person to earn that wad of cash.

Thinking about it this way, cash and bills are a rather poor technology in our digital age. Paper bank ledgers are better, digital bank ledgers are much better, and decentralized ledgers are (might be?) even better still, because they contain much richer information about the history of exchanges. Indeed Blockchain ledgers are organized this way: there is a public record of all transactions and account balances can be deduced form them. (The fact that Ethereum explicitly tracks balances of accounts is in fact unnecessary and possibly useful only for optimization reasons.)

Returning to the incorrect paradigm, as humans with our proclivity to focus on nouns, we think much more readily about account balances, the digital analogue of tokens, than we think about histories of transactions. But an account balance is nothing more than a summary of a history of transactions, and so an account balance is just the output of running a function on a history of transactions.

This function seems simple: subtract outgoing transfers and add incoming ones. If we had a ledger that didn’t change this would be fine. But this ignores the dynamic nature of ledgers, where new transactions are constantly being added and those transactions require some integrity guarantee (e.g. no new transaction should be added that makes the net total transfer go negative) and violating transactions should be rejected.

How did this lead to the DAO hack?

I’ll again defer to this excellent recap of the hack for the details, but I’ll try to summarize the hack using the above perspective on account balances vs. transaction histories. The DAO contract checks whether certain transactions are legal by comparing them to a stored record of account balances (red flag!). If the transaction does not cause the account balance to go negative, then the transaction is permitted. After the transaction is created the account balance is updated to reflect the outgoing transfer.

The hacker was able to exploit two bugs where the transaction history and account balance fell out of sync. The first bug is subtle: the code allowed for a recursive call where two or more transactions are recursively created. Both transactions attempt to move out the attacker’s entire balance out of the DAO. When it succeeds, the inner transaction deducts the attacker’s previous balance from the DAO’s balance and sets the attacker’s remaining funds balance to zero. The outer transaction also completes, and deducts the attacker’s previous balance from the DAO’s balance. But the attacker’s previous balance was just updated to zero by the inner transaction! So after the outer transaction completes the DAO has not only transferred twice the amount it should have to the attacker, it also thinks it has more funds than it actually does.

The second bug is a glaring error: before the withdrawal happens there is a function call that looks like it’s supposed to reduce the attacker’s balance even before the transfer is created. This call is unfortunately mistyped to call a simple logging function instead. Had it been called correctly, the attack would have been prevented. (Although, looking closely at the code, calling the function that reduces the attacker’s code balance would have caused a different though less severe bug. It may have been a sloppy fix to this less severe bug that introduced the vulnerability.)

Suggestion 1: stop using account balances

The first and most radical suggestion is that developers stop using account balances in their code. Instead, checking whether or not a transaction is valid (e.g. there is enough Ether to transfer out) should always be done by looking at the history of transactions and recomputing the net total transfer into/out of an account.

This is admittedly computationally expensive, and in a resource-constrained environment like the EVM may be a non-starter. Nevertheless perhaps this philosophy can help guide future development of the EVM or successor technologies.

Suggestion 2: create SQL-like commit/rollback functionality

This error would also have been prevented if transaction creation and account balance update were atomic: both succeed or fail together and at the same time. This is a basic standard notion in applications like databases and surprisingly absent from the Ethereum spec, given the importance of data integrity in financial settings.

In this particular case, wrapping everything in a transaction would have resulted in a final commit that tried to create multiple transactions and deduct the total amount of all the requested transactions from the account balance, which would have been rejected since the account did not have a sufficient balance.

I see no reason why the EVM couldn’t be modified to support such functionality. Maybe it’s even possible to support this functionality at a higher level without modifying the EVM spec. The protection it provides is orthogonal to some of the other tools that people are proposing (strong type safety, formal verification, etc.) so they should all be pursued, but I would guess that this is both simpler and more user-friendly since most people with some dev experience have used a database commit/rollback before.

Update: Thanks to edmundedgar on the reddit channel for helping me see that the above suggestion about commit/rollback isn’t really appropriate, since the EVM execution environment is single-threaded. The problem is not contesting threads, its re-entrant calls on a single thread.

I think a related solution would still help though: create a standard library for managing transfers of balances (could be both for tokens or Ether). An example transfer function might do the following: keep track of a “transfer active” flag per account, and when a transfer into/out of an account is attempted, first check the flag. If it’s on then throw an error, otherwise perform the transfer, update the account balance, and clear the flag. In this case, if the receiving contract attempts a re-entrant call, the flag will be on and an error will be thrown, thwarting the attack.

Final notes on terminology

I’ve avoided using the word “database transaction” (which is what this kind of commit/rollback mechanism is doing) since the word transaction is overloaded here. Any suggestions on how to refer to this idea without using the word transaction?

I also think it’d be nice if we had a simple and precise word to capture the notion of “exchange of money”. I don’t have the best words, but someone who has better words may be able to come up with a fitting and memorable term that we can use to help put the exchange of money at the center of our thinking about financial transactions rather than the money itself.


Share

In bits we trust(?)

/ Leave a Comment
Share

The Ethereum DAO hack has rocked the Blockchain world in the last few days, calling into question the security of Ethereum (the #2 Blockchain technology after Bitcoin) and with it the notion of smart contracts in general. Technical details of the hack can be gleaned here and for some background you can check out the very (too?) slick DAO website here.

In short, the DAO was the most hyped application of Ethereum (which is still in beta, mind you) and which garnered ~$150 million worth in Ether in funding, and was touted to be the first example of a distributed organization governed by code rather than people. It got tons of press and that exposure makes this failure all the more embarrassing. The hack exploits a bug in the code of the DAO and has allowed the attacker to drain $50 million worth of Ether (Ethereum’s base currency) from the DAO. It’s worthwhile to note that the hack affects the DAO but not the underlying Ethereum Blockchain, which remains secure as far as we know.

Instead of dwelling on the DAO’s failure and the possibility of a fork to save the lost ether (lots of chatter and Schadenfreude on reddit on the topic right now), I’d like to take the opportunity to focus on some lessons we might learn and suggestions for moving forward.

Too smart for our own good

The idea of smart contracts sounds extremely appealing: contracts that trigger actions on their own when predetermined conditions are met and whose terms are as hard and concrete as computer code and published to the Blockchain for all the world to see. No more hiring expensive lawyers to debate what that comma means, no more getting papercuts flipping through 5 inches of riders and addenda, no more yelling on the phone arguing about whether or not certain criteria were met.

This is a romantic notion that appeals particularly to computer scientists, for we see the world through our computational lens and are inclined to view laws and contracts as nothing but programs written in fluffy natural language. Stripping away the fluff and crystallizing the underlying intent in code, removing all ambiguity, seems like the natural next step.

In reality, the fact is that crystallizing the intent of contracts and laws into code is extremely difficult. Even moderately complex contracts are difficult to formalize and that complexity opens the door to misimplementation.

One approach to dealing with this is by flipping the picture on its head: we can hold the code to be the contract, and hold that there are no guarantees except what the code provides for. This was the position of the people who created the DAO, at least before it was attacked. But that is a sandy foundation for a financial network: if we really believed that, then there has been no exploit, the DAO is functioning as it was intended to, and we should pat the lucky chap who now has $50 million extra worth of ether on his back. How many people could you convince to use a network with that philosophy?

Beyond malicious hacks, this position also means that contracts cannot be patched when bugs are discovered. After all, maybe the reason that I accepted a contract in the first place was because I had spotted a bug and intended to use it to my advantage. By patching the contract you’re changing our original agreement.

It seems therefore that we must separate the implementation of a contract as represented by code from the abstract content or meaning of the contract, and that the former can and should be adjusted to reflect the latter if any significant discrepancy arises.

Of course, the content/meaning of a contract is often not totally well-defined, which means we’re not completely escaping the world of natural language and human ambiguity. This seems unavoidable (especially given the next point about dealing with the real world), but even with this caveat smart contracts via the Blockchain retain some advantage: automatic execution, public auditability, and improved (if not always complete) precision.

Reality bites

A much more important problem is the fact that contracts live in an extremely complex and uncertain environment: the real world. If you’re a programmer, you know that writing programs for complex environments is much harder because the number of things that can go wrong grows dramatically. While we know we’re supposed to think about all the corner cases and figure out all the contingencies, the fact is that even in a relatively well-understood environment like a computer network we often forget or fail to add in all the error checking and handling that’s needed to make a system truly robust.

Now compare that with the chaotic mess that is the real world, where unexpected things happen regularly that you couldn’t hope to have accounted for in your contract: a hurricane might wipe out swaths of New York City, ISIS might attack Turkey, someone might discover the cure to AIDS, and so on. Crazy events happen and there will undoubtedly arise events that a codified contract was not intended to handle. This will lead to disputes about whether the contract is performing according to its original intent, and we quickly fall back into a world where everyone is calling their lawyer.

Contract stewards

With the above observations, one proposal I’d like to make is that we institutionalize the idea of a contract steward and make it recommended or even mandatory for all but the simplest smart contracts.

The steward’s responsibility is to intervene in the event of a bug being discovered or if an event arises that’s not covered by the original code of the smart contract. The actions that a steward can take may be restricted by the original smart contract (e.g. an expiration date after which the contract state is immutable even by the steward), but should be rather broad in order to be able to handle as large a range of contingencies as possible.

One key question is who should play the role of steward? This can and should be decided on a contract-by-contract basis. The creators of the contract can designate a steward that they all trust, whether it be a friend, a court, or a company set up for such purposes, similar to an escrow service.

We see that this is already happening in an ad hoc way in the DAO debacle: the Ethereum developers are proposing a fork precisely to prevent the ill-gotten ether from being spent, and are thus playing the role of steward in this crisis. While the Ethereum devs may be the natural party to appeal to in this case because of their prominence, it’s not really clear if they’re neutral (some are invested in the DAO) or suffer from other conflicts of interest.

If we could institutionalize a neutral 3rd party steward in contracts, we would have a systematic response to hacks and bugs and more importantlywe may dissuade attackers from exploiting bugs because they know that their attack can be reversed.

Finally, this change is “easy to implement” in the sense that it does not require any technical changes to Ethereum itself. Rather, it requires only that new contracts be written to designate a steward and grant them the ability to intervene.

Some will cry that this is antithetical to the spirit of the Blockchain, and truly it is, if one insists that the Blockchain is about total and non-negotiable decentralization. But it seems like sacrificing this bit of ideological purity is worth it to escape the grip of immutable buggy contracts, which if allowed to persist will kill the appeal of smart contracts altogether.


Share