 All right everybody Welcome, thank you for attending my talk hacking cryptocurrencies My name is Mark Nezvit. I'm an application security engineer at Coinbase In the blockchain village at DEF CON you all probably know Coinbase But for those who may be unfamiliar Coinbase is a digital currency wallet and platform where merchants and consumers can transact with cryptocurrency Some of my recent priorities at my job have been security support for systems that integrate with cryptocurrency networks for instance our hot wallets and Second has been security assessments and mitigations for supported digital assets As you know you may know Coinbase is adding a large number of assets to the platform and we review each and every one of them for its security qualities I'm going to have two main sections to this talk first. I'm going to talk about what I mean by hack when I say hacking cryptocurrencies Second section. I'm going to talk about 51 percent double-spending attacks I'm going to walk through some real-world examples of this and then I'm going to talk about some observed patterns and characteristics of the attacks and the attackers that we've been able to uncover First what do we mean by hack? With every new technology a new exploit vector is born new processor developments like speculative execution or out-of-order execution We're exciting boost to processor speed and also enable the famous specter and meltdown vulnerabilities Blockchains are no different. They represent a new technology, which means there are new ways to hack. I'll explain one of them here Most everyone here is likely familiar with the CIA framework for security see for confidentiality I for integrity and a for availability Examining a system for these three properties can give a great start into understanding the security and threats against the system I'll describe how 51 percent attacks work in a bit more detail But for the time being it's important to realize that a cryptocurrencies a network of nodes that communicate to one another according to a protocol The nodes on the network store a copy of the blockchain Which is a public shared database and the network protocol allows nodes to communicate state information about the blockchain Nearly every blockchain has an authorization model based on public key Cryptography the state of data in the blockchain can usually only be updated when the proper digital signature is provided an example This would be sending bitcoins from one person to another the sender must authorize this state change by signing the send transaction a Wallet is an example of software that performs this action meaning that it's built on top of the blockchain Wallets hold private keys and can submit transactions to the blockchain as you could realize because the blockchain is a shared public database Anyone can choose to build a wallet application on top of a blockchain So they're a wide variety of wallets. They have their own CIA to examine Many of the hacks you hear about in the media are a failure of confidentiality in the parts of wallets Peter mentioned that just a few minutes ago in his talk If the private keys are leaked Anyone can authorize transactions for actions controlled by those keys Another interesting wallet failure mode is integrity failure Peter also mentioned this if an attacker can manipulate the recipient of a transaction prior to signing There's no need for the attacker to have access to the private keys. All of this is pretty standard application security work There's a lot of history in how to secure these types of systems a large component of my day-to-day work is doing that ensuring confidentiality and integrity in these systems For completeness, I've given two examples of availability failure that might happen These aren't really hacks because the attackers don't really get the funds But this talk is not about wallets. I'm talking about hacking cryptocurrency itself a new vector Let's see where the CIA framework can get us there I define the blockchain as a shared database. Thus, it's entirely transparent and there really isn't very much confidentiality Jumping down to availability This is also not really going to be a focus because it's not a huge concern for anyone except for the protocol designers This concerns matter concerns about the availability of blockchains. I've driven much of what's known as the scaling debate If protocol design makes the resources required to run and participate in the network too expensive It may impact the availability of the information which could have many negative impacts on the network Violating the integrity of a blockchain is what I want to talk about. It's a way to hack cryptocurrency The integrity has become recently become a bigger focus across the industry As I mentioned before I work for Coinbase a major cryptocurrency exchange Exchanges make an ideal target for all kinds of attacks, but especially 51% attacks, which I'll dive into in the second part First off exchanges hold a lot of cryptocurrency on behalf of their customers. That's an obvious enough reason for them to be good targets There are other characteristics though For instance liquidity and volume being able to trade one cryptocurrency into a different one can be very advantageous to an attacker speed Exchanges often credit funds to attackers on a relatively short time frame and allow for nearly instant sends an attack could therefore happen very quickly Remote interaction an attacker can execute many of these attacks from across the ocean perhaps from North Korea and In some cases anonymity I want to take a second to talk about this many popular media descriptions of cryptocurrency seem to describe it with some Magical anonymity anonymity qualities, which it doesn't have this is especially true If you have an authenticated session with an exchange such as Coinbase Coinbase strives to be the most trusted exchange in the entire cryptocurrency industry and as part of that We're heavily regulated and a large part of that regulation involves the lengths We go to ensure every customer on our platform has gone through KYC AML KYC know your customer so we know their identities and that's important for AML anti-money laundering any exchange That doesn't have these strict requirements would obviously be more attractive to a potential attackers so if you can find some sort of If you can find some sort of vulnerability whether that's subverting a protocol or a more traditional wallet style vulnerability As I described earlier this makes an exchange a great target so 51% double spend attacks as I mentioned before a blockchain is a shared database stored by all nodes on the network and accessible to anyone For this database to be useful there must be a way to update it Blockchains are append-only databases and are updated in batches of transactions each batch of transactions that's added to the blockchain It's typically called a block So you could visualize the blockchain like this With the expectation that a block n plus one would shortly be added But that raises a question who defines block n plus one the database is shared and distributed So there must be some way of coming to consensus among the network participants about what constitutes this block The answer to this question is that it depends on the cryptocurrency This is one of the major defining characteristics of cryptocurrencies and a lot of new cryptocurrencies have innovative methods for adding to the blockchain For instance Ripple and Stellar have a concept of validator nodes that use a voting consensus protocol to determine which transactions are in this block EOS Goes through regular elections where these nodes are known are called block producers and they take turns defining the block Tezos and cosmos the node chosen is based on its stake. It's a proof-of-stake network and stake is the proportion of network funds own Lastly Bitcoin and Ethereum The node that first successfully solves a cryptographic puzzle defines the block is as known as proof of work It's known as proof of work because the solution to the cryptographic puzzle has to be brute-forced Which takes considerable computational effort. This is called mining Mining a block is when a node discovers the solution to the proof-of-work puzzle Here's a key fact about proof-of-work networks Anyone can bring their computation to the table and if they produce the valid block they have extended the blockchain Since this is a distributed and permissionless way of extending the blockchain It's possible that the network will encounter multiple versions of the blockchain To resolve these versions and reach consensus on a single version the network deems the version with the most work To be the canonical blockchain. I'll explain what I mean by that This diagram shows the blockchain tilted 90 degrees with the block separated block n plus one will be added on top of the other blocks as Before each block contains some number of transactions Suppose a node with its computational power solves the cryptographic puzzle Mines a block it broadcasts the block that solves this puzzle to the network and all transactions in the block are added to the Canonical history of transactions that is added to the blockchain But suppose the second block is found simultaneously How does the network decide which block contains the transactions that are to be added? The rule is that the nodes on the network define the series of blocks with the most work as the canonical history So if either of the two blocks gets extend another block extending on top of it There will be more accumulated work on that branch which makes it the canonical blockchain This means that there's never a case where a block is truly finalized on the chain If enough work decides to extend from a different block once that branch has outworked the rest of the chain It will be the canonical history The situation on the slide is called a reorg short for reorganization And the great out blocks are known as orphan blocks, and they're not part of the blockchain Let me repeat a key facts any actor that can outwork the rest of the network is the sole arbiter of which among all Possible valid transactions are the ones that are added to the canonical history So if there's any kind of network instability where blocks were not always immediately shared with the network after they were found Or if some actor was deliberately holding back blocks that had been discovered We could see something like what's shown on this slide where the blocks on the left are hidden in from the network But if they were shared and made public the network would switch over to these blocks and define them as the canonical chain Orfiting all the blocks that were previously the most recent additions to the chain Because of this potential for instability of the most recent blocks Anyone receiving a transaction should wait for several blocks to be found after the block that contain their transaction To lower the chance that the block containing their transaction will be orphaned An analogy that I found interesting is that the most recent blocks are like recently fallen leaves in the fall They can blow around and change and shift after a while They might get waterlogged and not move very much and after even longer They'll decompose into mud clay and eventually rock you can adjust your risk by Selecting the number of blocks that you wait until you consider a transaction finalized This is known as the confirmation requirement and each recipient of a transaction decides on their own level So imagine we had the following situation where Coinbase supports a fictional coin McCoyne abbreviated MUH Suppose the confirmation requirement for MUH is three blocks Coinbase also supports BTC Bitcoin and MUH trading Any customer of Coinbase could have the following intention create a transaction T that sends coins from the customer's wallet to Coinbase Wait for three blocks after which Coinbase will consider the transaction finalized and Coinbase will credit these funds To the the customer's Coinbase account then the customer may want to sell the MUH for BTC And then send the BTC wherever they like off-site off the platform This is a completely normal pattern of behavior for a customer to take Let's imagine instead though that the customer is actually an attacker an attacker with a special ability to outwork the rest of the network The attacker creates transaction T sending some amount of MUH onto Coinbase Suppose T is quickly included in a block by some miner on the network Simultaneously the attacker will create a second transaction T prime Notice the T prime sends the same funds that were sent in T address at a one And however T prime sends those funds to address a two T and T prime could never exist in the same blockchain together as soon as one is included The other would be invalid an invalid transaction because the funds were already spent The attacker begins to mine in secret and includes T prime in the secret block, but not T The space on the right with the gray background is local to the attacker and the network cannot see this block Remember that we've assumed the attacker can outwork the rest of the network Meaning the attacker can produce blocks faster than the rest of the entire network So in order to do anything with the MUH on Coinbase it first needs to have three confirmations because that's what it takes to be credited The attacker does not sit idly by and continues to secretly produce blocks The network also produces blocks, but unknown to anyone. It's not keeping pace with the secret blocks produced by the attacker Finally the network produces the third block three confirmations on the transaction The attacker is now credited with the MUH and can sell it for BTC, which could then be sent off the Coinbase platform So the BTC could be withdrawn It's out. It's now in the attackers control remember nothing seen publicly thus far as anything out of the ordinary But now the attacker can execute the attack The attacker can reveal the blocks to the network these blocks have more accumulated work than the existing top three blocks So according to the network rules a reorg will occur the attackers blocks now representing the canonical chain The top three blocks that we previously seen publicly now become orphaned blocks They're no longer part of the blockchain and the transactions defined in them are now no longer part of the canonical history and Notice the T was in those blocks meaning that there's no longer a transaction to Coinbase in the blockchain anymore But the BTC has already been withdrawn. There was a withdrawal. There was no deposit aka a theft The ability to do this is directly related to how difficult it is for an attacker to overpower the network The more work being put into solving the proof of work puzzles on the network The more difficult it will be for any one entity to Marshall the resources and overwhelm the network Note that this the danger of this attack comes when you accept a deposit directly from the attacking entity in this example BTC was provided in exchange for MUH if the attacker can't get something irrevocable in exchange for this vulnerable coin The attack isn't viable. This is one of the reasons that in exchange is a great target for this attack liquidity The thing about 51% of taxes. They're pretty obvious if you know what to look for Each block in the blockchain is identified by its hash providing a unique fingerprint If the hash of block at height n changes from what it was before that block was replaced with a new block There must have been some kind of reorg Small reorgs shallow depth reorgs happen on a regular basis This is primarily driven by the fact that many nodes across the world are attempting to find blocks There is some amount of latency in that in the network So there will be race conditions where multiple blocks are found simultaneously and eventually only one will be in the blockchain however deeper reorgs Do allow for attacks if they're exceed the confirmation limit of the service You can inspect a reorg to look for the presence of t and t prime Two transactions are double-spends if they send the same money, but to different places They can't exist in the chain together, but they might exist in competing branches of the chain This is the smoking gun that a reorg is malicious money that was sent to one place It's effectively clawed back when the new blocks are revealed from the attackers point of view the attack has two components One is the ability to form the secret chain which requires majority hash power To the ability to create transactions t and t prime You're going to need some amount of the currency itself to do this and the more coins you have the bigger the impact of t and t prime An attacker is also going to need to select a victim. Obviously the victim must accept the currency But the victim has to provide something of value that they cannot take back once they realize they've been attacked So an attacker couldn't sell the coin for us dollars and transfer the us dollars to the to a bank account Not only would that likely expose the attackers identity, but the bank transfer can usually be reverted Cryptocurrencies that aren't vulnerable to 51% of tax. However, cannot be reverted This is another reason why cryptocurrencies exchanges make such good targets for 51% of tax. You can get cryptocurrency from them Also notice that this attack can be repeated indefinitely until the victim takes defensive action either by raising the Confirmations required on their service or simply shutting down their interaction with this currency We're going to walk through some real-world examples of 51% double-spending attacks The one will go into most detail is The 51% attack on Ethereum classic in early January was when this happened and because it's an asset that coin mix supports We had monitoring systems in place which alerted in real time to the attack allowing us to pause interaction with the blockchain I'll talk about how this attack unfolded The ETC network is mining its own business mining blocks as usual adding transactions to the blockchain Then all of a sudden seven new blocks show up out of nowhere And these seven blocks don't extend from the most recent block, but dig down five blocks back or fending four blocks 12 hours later and happens again six new blocks or putting five previously discovered blocks. I Called both of these transit or these these events practice attacks and the reason for that is because they were just reorgs There was never a pair of transactions T&T prime where the same money was spent in one place and the first branch to a different place in the second branch We hadn't observed reorgs of this depth ever on Ethereum classic and so it would have been premature to call these attacks But once we were seeing these we were we were alerted that something unusual was going on and and three hours later There was a very deep reorg 74 new blocks showed up all at once Orphaning 57 blocks and In these in this reorg there was a T&T prime where the same money was spent first to one place and then to another This was on a Saturday night. Our on-call engineers responded validated the alert and turned off ETC send and receive functionality Ethereum classic isn't the only attack Isn't the only successful 51% attack. It's the one we're most familiar with because it's the one we were closest to But I'm going to talk about two others that we've looked into closely BTG is Bitcoin Gold and VTC is vert coin, but those aren't the only other two. There have been others as well Some of the observed patterns we see from from looking into these three different attacks It's important to realize blockchains are public This means that a 51% attack is a pretty noisy attack. It leaves all kinds of good data for understanding the attackers I'm going to walk through just a few of the things that we've observed But they really only scratch the surface of what you can learn about an attacker These leave these attacks leave such a trail of damage You could say behind them that I don't think it would be very long before we're very good and learning quite a bit about attackers of this of this sort This chart shows all 17 of the reorgs that we were able to find in our research into the Bitcoin Gold attack And how much Bitcoin Gold was taken in each one of them? Notice the first two Didn't take anything This is vert coin VTC the first five took nothing And notice the first two in a theorem classic Also took nothing Remember what I said the 51% attack has two parts first being able to build the secret chain and then properly creating transactions TNT prime so what did the attackers do they broke the problem down into those two steps Made sure they could build an attract chain before they worried about building TNT prime even criminals need integration tests Criminals are also not perfect. These are the same three charts as before all in one slide You may have noticed that there were gaps when I first showed these to you They're a little harder to explain I think but as far as I can tell the attackers did a bunch of work in these cases to reorg the chain But they didn't put in a TNT prime. So they didn't cash in on any double spending For the first few attacks it makes sense to assume that they're practicing But once they've proven they can do that these just look like mistakes to me There's also an invest exploit decision that an attacker has to make Imagine yourself in the attackers shoes and you kind of have an interesting dilemma once you have the hash power to success successfully attack the network Any additional resources you have should be directed towards owning the currency itself to amplify the impact of TNT prime In other words the cost of the attack and the payoff of the attack are not functions of one another So as an attack progresses You're accumulating resources if you're being successful. Should you reinvest these or should you take them off the table? You know how I said blockchains are very transparent. We can observe the decisions. They made clear as day in the Bitcoin gold attack and In the vast majority of the vert coin attack attackers were mostly an exploit mode They have X amount of coins and every time they attack the network they perform a double spend they get X payoff But in the Ethereum classic attack and oddly in these first three vert coin attacks the attackers seem to be also in Invest mode this makes me think that the Ethereum classic attackers may have been planning to continue attacking because I would I would expect an Optimal attack profile include a period of invest followed by a flat period of exploit There's also something really interesting about the Ethereum classic data Notice how it steps up in pairs the first double spend was a time for a tiny amount also probably a test You can see in in at number four and five. They're roughly the same size But then the size roughly doubled for six and seven almost doubled again for eight and nine Stepped up significantly for 10 and 12 with 11 looking like it may have been another mistake and Then doubled again for 13 and 15 with 14 being another possible mistake It seems to me that the attacker was balancing investing and exploiting bigger and bigger payoffs every time But you never know when the party is gonna stop and so you want to take money off the table while you do it That's what this looks like to me This is also really profitable for the attackers You can see our estimates on how much the double spends were worth and our estimates on how much the mining cost would have been to perform These attacks you can see the profit margins. It's absurd. Our mining costs estimates were also fairly conservative We're also not even factoring the money made from the mining reward the attacker mine valid valid blocks Those blocks come with a mining reward just like regular miners would have gotten I'm not even factoring that in They're approximately on the same order of magnitude as the mining costs, however But the mining reward and leftover coins from the tack do typically get sold So attackers are cashing in on this in the case of Bitcoin Gold It looked like about 75% of the mining rewards removed shortly after the attack probably to an exchange probably to liquidate them Probably so that they could have coins that were unaffiliated with the attack Invert coin and Ethereum classic all mining rewards removed very shortly after the attack Analyzing the time of the day time of day that the attacks happen is another route to understanding attackers I've mapped the times of the attacks on this slide It's hard to draw meaningful conclusions from Bitcoin Gold and vert coin There does seem to be some clustering, but it's not too dramatic and they do have pretty much 24-hour coverage during the attack Ethereum classic, however Obviously has a time of day pattern in the attack and that one outlier that you see was actually the very first practice attack So it's it's a different case anyway If such a major pattern does emerge I would consider there to be two major hypotheses about what could be driving it The attackers preferred waking hours or the time zone the attacker considers most damaging to the victim probably at night Also note that the Ethereum classic attacks and most of the vert coin attacks happened over weekends again Probably because that is when it is most difficult for an exchange to respond When doing timing analysis note that these attacks can sometimes take hours for the attacker to build the chain Meaning the attacker doesn't have the full luxury of choosing their timing and maybe forced to work around the clock As an example the longest Bitcoin gold attack chain was 27 blocks, which probably took them over four hours to mine The longest vert coin attack chain was 310 blocks, which probably took them over 12 hours to mine As I said earlier the risk for 51% attacks is an accepting money directly from an attacker An attacker will want to find an exchange where it's possible to hide their identity from the exchange in the case of Bitcoin Gold Bitcoin Gold was delisted from bitrex shortly after the attack and the Bitcoin Gold dev team put out A statement claiming that bitrex was a victim of the attack and that explained their delisting I don't know who the victims were of the vert coin attack in the case of Ethereum classic three exchanges all put out statements Acknowledging that they were targets KYC can help prevent an attack another pattern the attack stopped after they're publicized cockroaches hate the light Bitcoin Gold is a three-day attack. It was publicized on the third day Vert coin was nearly two months. The day was publicized was the day of the last attack and Ethereum classic also a three-day attack the third day was the day it was publicized The last pattern I want to talk about is also interesting We've noticed that attackers commonly don't place their TNT prime transactions in the optimal blocks Considered the example of the very first Ethereum classic double spend attack that I talked about before Where 74 blocks were orphaned or 74 blocks did the orphaning of 57 blocks This would be the ideal block for an attacker to have placed their transaction T in It's the deepest block that was orphaned which would have given transaction T 57 confirmations at the time of the attack Instead T was placed 13 blocks higher where it only had 44 confirmations at the time of the attack Transaction T was not the deepest orphan block That means the attacker did the amount did the amount of work required to orphan a transaction with 57 confirmations But only for a transaction that had 44 This happens in the vast majority of the double spends that we've observed which is frankly very puzzling So let's review. I define hacking by attacking the integrity of the blockchain new technologies new ways to hack I described a 51% double spend attack then I walked through real-world real-world examples of these attacks along with the patterns that we've observed That concludes my talk. Does anyone have any questions? I'm sorry. I can't I can't quite hear you Yeah, so what I think let me let me make sure I'm understanding your question correctly when a block is orphaned Is that data lost? Is that what you're getting at? Yeah, then a node will store that data on its own So when a block is orphaned the node in its database will say this is an orphan block But it'll hold it and so you can still query your node for that data Your node had to be around at the time because those blocks aren't being shared around on the network But if your node saw it your node has it Yeah It's not exactly, but I think they're probably close especially in in Smaller market cap coins where general-purpose hard work can be moved to arbitrage the mining reward This is way more profitable if someone's gonna if someone is going to take a deposit for a Very weak amount of confirmations and give you something incredibly valuable for it It's just it's just so trivial to claw back the original deposit I don't I don't really think they are Yeah, well, yes when when they when they mine their blocks Those are just those are ordinary nodes and the easiest way for them to do it is so not even modify the software Just disconnect it from the internet mine your block You create your transactions and then once you you look locally you see that you've outpaced the main chain You just connect to the internet and then the reorg will happen by itself So coinbase has never lost anything in this sort of attack We've only seen one attack on a chain that are on a asset that we support which was the Ethereum classic attack We we noticed it immediately because we were monitoring for this and we immediately shut down so the very first there were Can't remember the number there was something like 15 of these reorgs with the Ethereum classic the very first one That had a double spend we alerted and we shut down so all the subsequent ones we're offline and they couldn't have attacked us Okay Literally So I'm not I'm not sure I understand your question I think you're saying if there's a delay on withdrawing from the exchange that makes it harder to attack the exchange And you're definitely correct that if there is a delay in withdrawing the exchange Because then the attack could potentially be detected and before the funds left you could put a hold on them But that's not true in all cases you can transfer on to an exchange sell and transfer off And that is a very common pattern and without without these long delays delays or something It's one of the pain points for a lot of customers in the cryptocurrency industry They don't want us to and wait for five days for seemingly no reason So it's it's a constant balance to get that number right and but the faster it is the better the opportunity for an attacker Any other questions What's what's that Once the attacker transfers out of Coinbase it's out of our control So then if they can claw back that original on send then they got away with it essentially Which hasn't ever happened to Coinbase just to be clear Any other questions? All right. Thank you everybody