 So yeah, my name is Jake and we're gonna talk about smart contracts and in particular how to sort of detect what ether scan calls Internal transactions. This is notoriously hard sort of thing to do and we've heard from lots of people that it's challenging So we just wanted to kind of present our experiences with it issues We've seen and how to sort of handle these safely and how to prevent you from having similar sorts of problems in the future Like you said on the cryptocurrency team at coinbase. We are the team that runs our blockchain integrations for all the currencies hot Wallet cold storage sends receives. We're hiring if that's interesting to you. Just come talk to me afterwards So yeah, we'll get started though. And so who should care about this stuff? These are kind of the primary three we see here is like exchanges wallets and explorers The highest risk comes to exchanges, which we'll talk about but really matters for everybody And anybody, you know building on a theory and the reason is because if you're sort of accepting deposits of any sorts of Ethereum, there's different ways it can happen And so your primary responsibility is to take a transaction and sort of turn that into a set of Debits and credits to accounts or to update balances or whatever you're trying to do here And so if we take a transaction like this, we're gonna go here and you know inspect it and say, okay We should credit Alice one ETH and we do that But the risk here is, you know, what if that's not actually what happened, right? What if we shouldn't have credit as Alice's ETH and there's some sort of bug in our code? This is what comes up with contracts because it's not a straightforward thing to do And there's a lot of nuance to how to handle it. And so what this can lead to is a couple of different things Invalid data is sort of the first one here, right? Just having bad data showing a wrong account balance is Not ideal. It's not the worst, but it can lead to pretty bad experiences If somebody thinks they have 10 ETH and they try and send that and then it just consistently fails They might switch to a different wallet And if you have systems building on top of this, if it's some sort of API and you have downstream customers Then they're gonna get bad data and they're gonna have to update Accordingly their database and that's gonna be pretty challenging and there's not really good ways to handle that So you want to avoid this sort of as much as you can And like I said, the real risk is actually to exchange us here with lost funds because what they can do is this type of attack Where you send in Ether, but you don't really it just kind of looks like it you can trick the system And then they can sell that for some other currency and send it off somewhere else or they can You know just send off ether and in the exchanges cases a lot of them have pooled wallets where everybody's funds are together So if you send in and you send back out, it's probably not gonna be the same ether It's gonna be something different and so you can actually just steal a theory in that way and continue this until you know You're caught and So if you have these types of mistakes, this is the types of headlines you might see written about you It's it's not good if you see hacker one reports like this You also don't want to see these kinds of things and this is what we sort of saw at some point or in the year and Yeah, you don't want that and so what can go wrong here? And like how do we fix it to start that we need to talk about how a theory and moves So as I'm sure most of us are familiar with everything happens in a block Then you have a set of transactions and in those transactions a theorem can move But within the transaction it's not just one way you sort of have two different types here You've got what we can call account transfers It's just a standard transaction Sending from one account to another and you can also have this other bucket here Which is like contracts and contracts can be pretty clever things which we'll talk about here But the account transfers like I said are pretty straightforward This is what you'll see on ether scan and this is kind of what you'll get back from your like node If you're just asking hey give me a transaction It's gonna look a lot like this and calculating, you know, how much value went to where there's a from there's a to All that's pretty easy to do and there's not really much risk here If it says a transaction happened in this format it did and you're pretty good to go But where it gets complicated as contract transfers, they might look like this So now instead of you know, it's just sending to one user You can send to sort of n amounts of accounts and it scales up and then what's really sort of Difficult about these is that this is custom code people right So they have a lot of room outside of just what the protocol allows to trick you and to specifically target like issues in your system and You know pack you because of it. And so there's not a great API for this What you have to use is this API called the tracing API Which is fairly low level and not that easy to parse and it looks a bit like this You're not gonna be able to read it, but it's pretty complicated and this is actually a simple one This is you know, basically one of those sends where I showed like 10 in the previous slide So it gets more complicated. There's a lot of nuance here But if we started to think about inspecting these and seeing what it looks like if you zoom in you'll see like There's a from and a gas and a to similar to a transaction But if you were to take it and just treat this as such you'd be in for not a good time and So we're going over a few examples of here different types of problems that can happen and And the right ways to sort of handle this and so on this left side We have what we're gonna call traces here. There's different types of calls and we see one We send Alice one eith another to Bob and if we just turn that into credits and debits accordingly we would do that and The issue here is like I presented earlier It doesn't actually happen that way in this transaction and it's because we're missing something So not everything's gonna be on the slides here and we're gonna sort of bring those in over time And so in this case what you actually need to look at is this thing called the Transaction status, which we'll see here. So this came with an upgrade I believe last year I think of the Byzantium upgrade They added this new opcode and they also added the status to this thing called a transaction receipt And so when a transaction is including a block the blockchain will generate this thing called the receipt which tells you It's similar to like a store receipt. It's just you know Here's the final state of what happened And so if you didn't use all the gas or it failed for some reason that kind of information will be there And so in this case we see that the receipt is a failure And so what that means is actually none of the transfers that we might have that might have happened if it Didn't fail can actually happen or didn't happen and so accounting for that in this example We see here that nobody received any and we're good to go So for example to this is the same sort of scenario here Except we do see this thing that says fail on one of these calls And I didn't really describe the calls yet But a call is just this sort of it's this type of a trace There's different a few different types and so these sends are also calls But these ones are other ones that may just like call some function to you know log something or that doesn't actually transfer anything anywhere it's just you know calling a contract and so Generally, you might ignore these if you just care about sort of your customers information and So learning what we just learned we do check the transaction receipt status when we try and parse this and we see it's a success So we credit Alice and Bob And if if you sort of caught on from the previous example it didn't actually happen But what's interesting about this scenario is it's not that nobody received any it's that only some did So this transaction receipt status doesn't tell you everything There's this possibility of like a full failure where the transaction receipt is a failure But there's this partial failure states as well and these can get more complicated than this And so in this example what we really need to do when we're looking at these traces is turned it into a tree So it's actually if we can see this like hierarchy here We have this call at the root and then we have you know Alice receives one and then a few nested calls And then one of them is a failure and what this looks like in the trace is there's this thing called there's an error on it and If it has an error that means it failed and so when you put it into this tree format You can see that the send the Bob is actually within that part of the tree And when it's there what that means is that that doesn't actually happen and you shouldn't create any credits or debits And so accounting for that we give Alice the one ETH and we're good to go So the last example here looks entirely the same as the former one So there's even more here to kind of look at and so in this example if we did the same thing we credit Alice and then that's wrong and In this time Nobody should receive anything and So there's another thing missing here. So when we look at these calls, there's different types of calls There's not just one there's all sorts of different ones but in particular in this example, I'm using one called a Delegate call So we need to pick out the specific types of calls that can actually Transfer a theorem around and sort of either ignore the other ones or treat them differently And so in this case a delegate call Without going to sort of details of what exactly it does it looks identical to just sending a theorem somewhere except It doesn't actually do that and so in this example when we account for that We do see that Nothing happens So that was a little bit like how to handle these and it's a tricky thing and it could get more complicated in the future That's like it today, but a lot of really clever things are happening a lot of cool stuff People want to work on and it's really hard to get this right and these things aren't that documented either and so we got to figure out a way that even if we don't know the answer, how do we sort of Make sure we're not exposing ourselves to these issues because if you've been in many of the talks today So many of them are based around contracts right identity and all sorts of things are just contracts sending everywhere And so you have to be able to support these but many exchanges or many wallets Don't because of the complexity around this and because of the risk in particular to exchanges of getting like hacked and actually losing money And so the kind of answer here is this type of sort of detection and response And the first step of that is just acknowledging that you don't know everything Things are going to change and especially if you're working with multiple blockchains and multiple currencies That kind of stuff is just some of it's gonna slip by some little note in a release notes of some upgrade of your software Is going to change something and you may not realize it And so you just want to take a step back and say, you know, I might not know everything if you don't well Like how do you do this? Like how do you make an answer here? And so one way to do this is sort of cross-check your data But there's a lot of people building there's a lot of people you can talk to about this But really if you can automate this that's sort of the ideal here if you can cross-check your data and say Oh, well, I think this happened and then this other thing says it doesn't and it's a source you trust Then maybe you shouldn't do that. And so what you should do there is Actually like fail securely so rather than failing like open and letting some sort of payment go through and you know Credit an account and allow that to be hacked just set it aside Like put it in a queue alert somebody on your team They can investigate it and see what's going on and then they can determine if it's actually a bug or if it's not And so this is actually this is the kind of stuff We've done now and it's actually really helpful even outside of this like since we've implemented we actually haven't seen anything like this and But what we have seen is just little things being off here and there You know some calculation of a balance is slightly wrong or some type of send we didn't so or some type of receive we didn't support And it's just like oh it's off and it's like cool It's like it's alert just that something's wrong and we get to go in and fix it and figure out why and improve our systems Without ever affecting any of our customers, which is even even better that you get alerted of things that are wrong You're just like oh cool. I can fix that and so We'll kind of put things in a takeaways here and sort of sum all this up So the first step is check the receipt status. It's it's easy to check. There's apis for this. It's a Boolean field You know, it's true false. Basically. It's numbers, but So just check that if it says it failed then Not all the transfers happened the nuance here is like you do still pay for the gas here Like failed transactions are still included in blocks because somebody had to execute it all the miners and nodes You need to execute the transaction to the failure to validate it. So it still costs like resources So even though it failed you still need to check it and they still do actually pay the gas for it but but none of the transfers and The second one here is to parse the trace as the tree It's there's information in the way the trace is formatted and the traces I sort of showed here that I'm describing or parodies get has a tracing API like this to now But it's it's they're not the same. It's not a standardized thing And so you need to reference sort of gas documentation for how it works there I imagine it's similar, but you need to take this flat list They they provide you and sort of structure it into some sort of tree and then once you do that you can inspect These sort of aired sub trees and you like filter them out or mark them failed or you know, whatever you want to do there Thirdly we want to reject these like delegate and call code calls These are two different types of ones that kind of look like they transferring things in but they actually don't and Then if you if you kind of take the the earlier point about failing safely here What you really should do here is actually just select the ones you know do it So this is similar if there's an upgrade you don't know about if there's some sort of change At least you know these ones work I didn't mention these other ones, but these are like creators for creating contracts self-destruct for destroying one and Rewards for like block awards. These are the kind of things that can you know credit Ethan like other ways or like send it and so you just if you select those and handle those cases Appropriately, you know, you're sort of at least reasonably safe and then if you implement this next point Don't trust until verified right verify your data check with other sources You don't have to like trust it completely, but at least it's sort of a signal that something's wrong Right, don't just take that and write it to your database. You're like, that's the truth But what you should do is at least look at it and be like all right Something's weird here and also if you're getting a lot of false positives, you know, maybe pick some other source And yeah, that's it for today So I hope you all stay safe out there and hope everybody we learned something here and we've got I think Some time for questions about four minutes. So if you have any