 Well first I just want to say thanks for letting me come here and speak today. I see a lot of friends in the audience and it's just been amazing so far. And at Coinbase, we don't take adding a new currency lightly and when we added Bitcoin Cash we wanted to make sure that we offered a fantastic experience for the Bitcoin Cash users out there just like we do for our other assets. So high level, we had some things that we needed to do when we needed to incorporate Bitcoin Cash. We needed to make sure that obviously our base functionality, buy, sell, sending and receiving and all the widgets that we use for a third party or API integrations were available for Bitcoin Cash on day one. We also needed to make sure that our mobile applications and GDACs were ready for us to do this launch and that we could do this together across our entire product line as there's dependencies within that chain that absolutely needed to happen. For instance, Coinbase and GDACs, they use liquidity together so GDACs has to offer books for Coinbase to use so we can hedge our positions and make sure that people can buy and sell digital currency without dealing with large price fluctuations. So first I want to kind of announce the players, give you a little insight as to the tools we use at Coinbase and most of these people probably haven't heard of before. So the monorail, that's what we call our main code base. It's a Rails code base. It's what powers Coinbase, your API services today. It powers all of our mobile apps and mobile API integrations. And then there's what we call the Scotsman. The Scotsman are the things that look at UTXOs and our blockchain specific. So in this case, we're gonna be looking at Wallace. Wallace is our Scotsman for managing all of our Bitcoin-like blockchains. That includes Litecoin, Bitcoin Cash and Bitcoin Today. It's a single code base and is also written in Ruby. And I'll show you some of the tools we use there in a little bit. Then we have ChainD. ChainD is all of our syncing solutions for full nodes. How do I deploy full nodes safely? How do I make sure I can back up state just in case something goes wrong? Recovery tools so that we can make sure that those chains are available quickly to our customers and avoid any level of downtime that we have within our product. And then it's all of our deployment tools. Codeflow and Heimdall are our consensus-based deployment tools. So Heimdall checks every pull request, looks at diffs, makes sure that enough people have looked at every bit of code that is going out into our production environments with different requirements based on how sensitive those code bases are. So if it's something that touches a private key, we're gonna need a lot of people to sign off on those changes. Versus something that's trivial like a front-end website change, which may require very few approvals. Then we have Codeflow, which automates all of our deployments. Everything in our fleet is redeployable at any time. In fact, we roll our fleets of servers regularly to make sure that our deployments are always fully automated and up to date and ready to deploy just in case we have failures within our AWS environment or any of our deployment environments. So we had some unique challenges with Bitcoin Cash that we had to kind of work through. First off is UTXO management. There's a lot of companies out there that kind of find other people to write tools for their UTXO management while we do all of this in-house. All of our UTXO management is written by us and is code bases that are about five, six years old and quite mature. But because of that, we really can't rely on a third party to help us with those integrations. And in this case, this was the first time we were adding a chain that had a shared history. That meant shared private keys. That meant being able to handle cold storage requirements where I had to worry about what currency I was moving and whether remaining currencies was remained sat on those private keys and those UTXOs. So that was a unique challenge for us. We're also working within a million plus line code base with micro services and it is non-trivial for us to go through and audit all of the tools that we use in order to make sure that a deployment is safe and that we can protect the billions of dollars of assets for our customers. We also have several different teams between GDACs, the mobile team, our security teams, etc. We all have to be on the same page. Some people are managing our hot wallets and hot and cold wallet infrastructure. Some people are making sure that our iOS and Android apps are there and we need to make sure that everyone understands what API changes are being done and what breaking changes may need to occur and how to roll those out safely and do that with feature gates. And then all the deployment, working with our infrastructure team and making sure all of that is ready and that all the automation tools for bringing out our new wallets instance, etc., are all fully automated. That chain D is ready to be backing up the BCH chain and that we can extensively test that on test nets before we roll that out into production for our main customer base. And then we had to do all of this while we were experiencing record traffic. So if you guys remember the end of last year, things were kind of gangbusters. We had so much traffic and we wanted to launch new features. But to do that, when we had the level of growth that we did, there was so many refactors to our back end data stores, etc. That that really had a lot to do with making sure we could deploy this safely and not trying to rush it to market and watching our site crumble as we try to add features and make our users happy. So, UTXO management, an issue of time. So, since we had to sync the UTXOs, we had two different strategies that we could do. One is we start syncing UTXOs from Bitcoin Cash from the Genesis Block. Well, that would have taken me four to six weeks in order to do that and make sure that all of those UTXOs were being tracked for all the addresses associated with our customers. And that would have extended our timeline on delivery quite considerably. But we did do that and we started syncing that and we decided that, well, we need another strategy to do this. So we also had a snapshot of the UTXO set and we had the current UTXO set from Bitcoin as well. Well, what we found is rewinding the Bitcoin UTXO set was the safest option for us and that was the fastest, but we also were extremely paranoid. Being the biggest exchange out there holding some of the highest liquidity means that a mistake can cost a lot of money. And so, thoroughness is absolutely a requirement for anything we do. So we reround the UTXO state from Bitcoin. We then checked all the UTXO state that we had to make sure that balances were correct for our customers across multi-sig accounts, regular accounts, vault accounts, and we wanted to make sure that everything was perfect before we launched this as when you launch a new feature, a lot of people check it out right away. So actually one more comment. If you wanna see what we did for what tools we use internally, we have a gem called Bitcoin Ruby that does all of our signing and specialty logic for Bitcoin. We are a Ruby shop and the other parts of the UTXOs and the signing infrastructure were actually really easy. Doing sig-hash fork ID and the changes required for that, those took a day, whereas making sure our UTXO set was right took a lot longer. Another quite address uniqueness. So this was a unique challenge for us as Bitcoin Cash and Bitcoin shared the same address format. So we could no longer look at an address and say, that's Bitcoin or that's Bitcoin Cash. Like we could do with every other coin we had currently on the platform at the time. It's very easy to differentiate a Litecoin address from a Bitcoin address or an Ethereum address from a Bitcoin address. And there were assumptions in the code in numerous places that we could do this introspection and some of our back end models really were not ready for sharing an address format. Whether that came to uniqueness validations on the address itself, where they're no longer really unique and have to be represented for both chains. So that required significant code refactors in order to do that. And that's what led to currency interfaces. So when we first started at Coinbase, we were a Bitcoin shop. Well, the world has changed, now we're a multi currency world. But when we added ETH and LTC, we did enough to add them. We knew it was mature support, but we really hadn't designed it with extensibility in mind, having objects that really encapsulated everything that that currency needed to operate. And so we took the time with Bitcoin Cash to actually refactor all of our currency handling within Coinbase. We now have dedicated currency objects that talk about what accounts we use to calculate fees, how addresses are handled, how they're displayed. And basically anything that you could ask of the particular currency is now encapsulated there. And we have removed all case and if conditions from our code base. This was a huge win, but also took a lot of work. So you never go in and it's hard to predict how long that's gonna take. And there was a couple of dedicated engineers across different teams that made that a reality. And now during code review, if you see if ticker, that's rejected. That's not allowed. We are trying to make sure that we can always conform to a currency interface so that adding future currencies and forks in the future will be easier for us if we deem a currency as something that we want to support. So cross team collaboration, making sure that everyone is gonna be on the same page with these different integrations. That was really very important, is first off, we needed to incorporate everything on TestNet. And the TestNets for Bitcoin Cash, luckily working with the ABC team was really fantastic. They helped us get our TestNet environments up. And we had everything working for our developers, as well as our staging environments on TestNet. Where any developer within our stack could really go and play around with any features on the blockchain. Also required a lot of updates to our Docker Compose setup. We use Docker extensively in-house. Every developer has a Docker Compose that involves 15 to 20 services that get brought up because, well, we need parity and lots of nodes and all of these things. And we have mocks for some of them that are more expensive to run on the developer machines. But that required a lot of updates. And those updates in our Docker Compose format is how it sends it to our integration and testing environments that are based in CircleCI. So the same Docker flow that our developers are using are what also go out to our CI environments. And then those are also not quite exactly the same in production. But there is some similarities there. We do use Docker in production as well while Docker Compose is not quite how that's set up in our production environments. As it's not quite mature nor is that the use case Docker intended for. Also extensive feature gates. So we tried to make sure we could deploy code while we were writing it. And that means not exposing functionality to our end users before it was ready. And that meant a lot of additional code to manage features. Like should there be a hedger for this? Should this display for brokerage? Should the mobile teams be able to do this? And our feature gates are all based off of our Coinbase version for our Coinbase API. But also done on a per user basis and per group basis. Which makes it so that I can turn on features for my staff and for other people that are needing to test without exposing it to the public. Then we also had to make sure all the documentation was available. As we don't necessarily always work directly with the teams, although it's a very collaborative environment. But we need to make sure that GDAX and the mobile teams know exactly what is changing to make sure this could be supported. And we needed to give them lead time. As when you're dealing with application stores, you need to be able to know that you have a week before that's gonna actually be approved. So breaking changes really can't go out. And you can't assume that things are gonna just work with the existing mobile code bases. Although for the most part there wasn't too many headaches there as I've been very impressed by the mobile developers that we have on staff. Now testing, well, we hold billions of dollars. So testing is really important. We did a fully test driven development workflow for all of these features. And that meant unit tests, integration tests, functional tests, dev environments that we had invites to for our dev staff. And also inviting non-developers to come and play with those integrations so that we had less technical people that looked like our target audience to be able to test these integrations. Now, like I said earlier, we're a Ruby shop. We do a lot of front ends in JavaScript as well. We use React for a lot of the Coinbase front end. There's a huge JavaScript testing side that our brokerage team uses. I'm not gonna cover that in this talk today, as this is more about the underpinnings on the back end. And on the back end for us on the Ruby side, we use RSpec, which has been, is a fantastic testing framework for Ruby. As well as Capybara for full screen integration tests. When we have test failures that include screenshots and very detailed messaging so that we can track down those errors. We also have some of the most extensive logging infrastructure. We log terabytes of logs every day. Every dev environment, every API call, every round trip to our data stores, all of it is logged. And our Kibana logging infrastructure, you can actually read about on our blog one of our infrastructure specialist, Luke. He is fantastic. He is the brainchild behind Kibana, not to discount all the other members of our infra team. But he's really done an outstanding job so that we can find anything that's going wrong very, very quickly and effectively within our code base. And that we're not just kind of thrashing trying to figure out, well, why did this break? We should always have the metrics on why this broke. And exactly where those error points are. And we do have those due to a lot of hard work of our scalability engineers that was happening during a lot of the growth that we experienced last year. And as I said before, at scale, our number one priority is that when we launch these, that your funds are safe. And that the people that are trusting us to hold their assets, we don't take risks that could possibly do anything to risk you losing money. And on BCH rollout day, there was a lot of Bitcoin cash we needed to give people. A very non-trivial amount of money was handed out that day when we turned on Bitcoin cash. And we were really happy that we were able to do that as safe as possible for our customers. So I've noticed a lot of people like to buy Bitcoin cash, but I don't see my widget everywhere. So I spent a lot of time working on this and making sure that the Coinbase Buy widget from day one supported Bitcoin cash. And Ryan and Clemens were the first to incorporate this on yours.org. You can go there today and you can take a look at how that works. It supports non-coin-based customers up to a limit. And then if you want to do major orders, well, we're going to want you to be one of our customers. But this is probably the easiest way to buy Bitcoin cash. And it is very, very easy for anyone to integrate inside of a third party site. And we have partnerships with people like Metamask that have done this for Ethereum. But I'd love to see this adopted more in the BCH ecosystem and really take advantage of the tools that we've built to make BCH more accessible. There's a little screenshot of yours. Make sure I get a clue. So originally, this was done with a Litecoin integration. I was not very happy to see that. I know that Ryan needed something, but we needed to bring him Bitcoin cash. So we did this mainly to make sure that the use case for yours was fulfilled. At least that was my motivation. Then the talk wouldn't be complete about some of the new things that we've been working on with Bitcoin Cash. So Cash Adder is an advancement that we very much needed at Coinbase. We do get support issues of people sending things to the wrong place. This isn't an imaginary harm that people are trying to make up. It happens and people lose funds because they send it to the wrong place. And we try to mitigate this as much as possible at Coinbase to make sure that people can recover the funds that were sent to the wrong chain. And Cash Adder was a way that we could stop this. All of our default receive addresses today, our cash address, to make sure that people that are running BTC wallets don't accidentally send to the wrong place. And we wanted to make sure that we conformed to the spec. We worked with Shama Micro-President on making sure that we integrated it correctly. We tried to be forgiving of allowing with and without prefix so that it was a seamless upgrade and people didn't have a weird experience using the new address format. And we did keep support for the legacy format for the time being where we recommend Cash Adress. We use it for all of our new stuff, but people do need to send to a legacy address sometimes. They have a Trezor, they have a Ledger. And until some of these other companies in the ecosystem make sure that the tooling is mature, it was a little premature for us to just go Cash Adder. So give me a reason to remove legacy support and finish out the rollout of something that I think needs to be everywhere for the Bitcoin Cash ecosystem and we'll follow suit. And it took us a little while to implement. Unfortunately, because we use Ruby, when I implemented this, there was no Ruby libraries. So I went ahead and I found a simpler version of a Python library and I did the porting effort myself. We actually used that for Coinbase Commerce as well. And we wrote our own library support and I believe that there's someone working on open sourcing our integration of Cash Adress which supports both Testnet and the main networks. Although Reg Test is not currently supported, not many people are using it and it really isn't required for me to do my testing. So if you really need Reg Test support, let me know and I can work on that. And then as a conclusion, I know that was a little bit of a brain dump. It involved a lot of development teams, a lot of coordination between the teams and a lot of testing to make sure it went through. Our major time sync was the UTXO set to make sure that everything was perfect and that we could implement the safest possible rollout for our customers and the testing processes where we spent weeks testing the integration to make sure that we could bring this out safely. And if you want to do BCH purchases on your website, again, check out the Coinbase Buy widget. And there's one last thing before I close out is I didn't add this to the slides and I wasn't sure I wanted to talk about it, but there was unique constraints on a private key when it came to our cold storage as well. So when we restore things from a private key, we like to burn keys that don't have funds and that way nobody ever has access to a key that was ever used for cold storage or storage at Coinbase. But that assumption broke down for us when now we had two currencies on a key and now all of our cold storage magically had all this additional funds on it. So there was massive rewrites by our security team and our crypto teams to make sure that we could redo most of our cold storage infrastructure to support BCH and make sure to do that securely. And that took a lot of time and a lot of effort and a lot of very, very delicate programming to make sure we could do right. And we've been very happy that the feedback we've gotten for BCH has been very positive on our service and people seem to be having a good experience and I'd love to hear feedback from you guys on whether or not you've had a good experience with our wallet. You can feel free to find me on Twitter, I'm ZQuestZ on Twitter, pretty vocal there. And what I say there is not representative of Coinbase, that's my personal Twitter. But if you have any questions for me, feel free to find me after the conference. Thank you, yeah, give it up. Woo! That was awesome Josh, thank you very much. Make sure to follow Josh on Twitter at ZQuestZ and do we have questions from the audience? We have one down. Check behind you, you've got somebody coming behind you. Hi, I would just like to know, since you've implemented Cash Address, what is the impact that's had on the number of people sending coins to the wrong chain? Has it like drastically reduced? Absolutely, there was a dramatic drop off on missed sends. Now unfortunately not all exchanges have incorporated Cash Address yet. So some of the exchanges that have not done that only send to legacy, we still see some of those tickets. But we've been reaching out to anyone that was the cause of a missed send and try to encourage them to adopt a safe for address format. As it helps everyone, it helps the Bitcoin users, it helps the Bitcoin Cash users. This is a win-win for everyone and I just can't wait to see more people adopting what is ultimately beneficial for their customers. Yeah, one of our policy solution providers in Australia main argument against rolling out Bitcoin Cash is that they get heaps of burn transactions, people sending money to the wrong chain and I'm working with them to try to say, Cash Address is the solution that's gonna... Well, it's a solution for one way, but not for the other way. And what's nice is because it's the same private key and same infrastructure, you can actually mirror monitoring the addresses for legacy across both chains and recovery is actually quite simple, unless Segwit is involved. So the moment you have Segwit involved and someone sends Cash Address, sends a transaction to a Segwit address from BCH, there's nothing I can do. I'm sending them to some miner and maybe they'll be able to mine it for them and recover those funds off the public key, but there's really nothing I can do at Coinbase as a non-mining organization to recover those funds. So we have very substantial warnings on the site to tell people not to send it to the wrong chain and we try to make sure that we do that, but hardware wallets are probably the main blocker for us not removing legacy support today. Yeah, while we're on the topic of people sending money, like it's lost, any possibility of some love for the layer token users that have sent funds to Coinbase addresses? So you can definitely talk to our support department about specific cases and we do try to help people with lost funds. It's not something that we wanna do all the time. We try to tell people this could lose funds if you're sending the wrong thing, but in certain cases, if it makes sense, talk to our support department and they'll see what they can do for you. Okay, thank you. All right, come on people now, don't let Josh down. I don't know a lot about the passing of addresses, but one thing that's seemed strange that there's a lot of sort of passing and copying and pasting of addresses in a non-URI format, is there a particular reason for that and why are we not passing addresses around as URIs which are meant to create namespaces and sort of separate or disambiguate a little bit? So actually, all of our QR codes are URI based. We never give you just a raw address anymore, so if you're in our Coinbase wallets and you have a receive address, it will be a fully qualified receive address. When we display an address, unfortunately, we do sometimes trim the prefix for just display because we don't really expect people to type them. But when you copy and paste or look at a QR code, that should always have the prefix there for safety of sending, and I agree with you. It should, whenever possible, be a URI. We really, even if you have checksums on those addresses, nobody should be typing them. Yeah, I mean, if the copy and paste is putting the prefix on, that's fantastic, and that's a step in the right direction. And it would be good to see some other service providers falling suit. Absolutely. Hi there. Some of the Bitcoin cash developers have been talking about enabling exchanges to run multiple nodes at the same time to ensure that zero days won't bring down trading. How would Coinbase feel about that? So we're a huge advocate of running multiple nodes. We, in fact, do that for Ethereum today. We run both parity and geth nodes, and we will only do actions when they agree. And we report bugs to the underlying implementations when there is not consensus on the reading of the chain, because the chain's the same, but somehow we get different results. And I would love to be able to do that for more implementations. Today, we are using ABC for our implementation, but I would love to get XT and unlimited nodes involved, and that way I could provide a early warning signal if I see inconsistencies across those implementations. And I would love to work with the developers on making sure that all gets set up. We're gonna take that to the back. Quick question for you. We've talked about unconfirmed transactions. Any support for that on Coinbase? So I can't speak about future changes. I can say personally that I'm a fan of Zeroconf, and a long time ago, Coinbase had Zeroconf, and we offered that at one point. And then RBF kind of ruined that for us, and this idea of confirmations everywhere. I think that that conversation can come back up now. And especially as more protocol work happens, that becomes more and more of a reality. When I have better double spend relays, when I have better assurance that that is gonna hit, and that I can trust the Mempool more, whether that is proof of work-based consensus preblock or non-proof of work consensus preblock, I wanna see that research. I'm really excited to see that being pushed forward. And definitely, we wanna do what's best for our customers, and having customers wait for no reason doesn't make sense. Gonna be on Bitcoin Cash. Holy moly. All right, we got any more questions? Josh, just wanted to say thanks both to you and Brian, but I just wanted to know if you had any opinion. Roger asked Peter a question yesterday regarding how to market zero confirmation to the general public, and maybe better nomenclature from a marketing standpoint for educating new users, but also existing users. So, I mean, this is just a personal thing for me, is I think they're just transactions. I make a transaction. And when I swipe a credit card, I just consider it a transaction. And it feels like it confirms instantly, right? Nobody talks about 60 to 90 day settlement on a credit card. And reality is, is that you have a long time before a credit card transactions final, and you're trusting something way less stable than a 10 minute mem pool when you're trusting a credit card. When you look at standard charge back risk and is far higher than what I've ever seen calculated for double spend risk for most amounts of money. And therefore, I think they're just transactions. We have instant transactions, and they settle in 10 minutes. All right. Okay, is that it? I need another round of applause. It better be a good question. Hopefully. As I asked you yesterday, I would like to know if you're willing and able to work together to implement your service in countries like Colombia. Because we already, today, we don't enjoy your service. And I think this is crucial. So, I can't talk about exact markets. That's not my job at Coinbase. But what I can say is that we want to build the next generation financial system for the world. And that includes every country out there that is willing to deal with a regulated exchange that wants to work with the banks and governments in those jurisdictions. And we're always looking to expand our services to new places. And we're also looking for developers. So, if you have people that want to come out and help us build those next generation tools, banking integrations, new crypto offerings, those types of things, we are actively hiring. We also have protocol developers that we want to be able to fund as well. And that's not us dictating what they do, but supporting the advancement of the protocols out there. So definitely check out our blog. We really want to push things forward. And that means new markets. And ideally, I'd love to be in every country in the world. Thank you. All right, we need to give Josh one more round of applause for the good question. Okay, well. Really a question. I'm from Australia and I buy through Coinbase. It's my favorite app, by the way. The only thing is I've noticed a really bad buying experience for Australians. I know you don't have, that's not your area, but can you put in a word for us? I will put in a word, absolutely, because there's a lot of people in Australia that are big crypto fans. So I can definitely do that. And also report to us the issues that you have because we love getting that feedback and we can only improve some of these integrations with the correct feedback from our customers. All right, thank you, Josh. Woo!