 has like three questions and we've got like, now we're at 15. Hello everyone. I'm Dean. I work at Status. I'm a security researcher. I used to do the entire auditing thing. I haven't done it in years. It's going to be super interesting to see how this space has progressed and I have a really competent panel. So I think it's probably best if everyone just goes around and introduces themselves, maybe says what they're working on and what some of the most interesting things that have happened in the past year in the security space have been. Maybe events, encounters, changes, things which have progressed in the right direction or things which we were still seeing which we didn't expect we'd see one year old. Hey guys. My name is Gonzalo. I'm here representing the consensus diligence team we're part of the consensus but we're also squashing bots. We've been squashing bots for the past three years, I guess. I'd like to say that a lot has progressed but we were just talking outside about how it kind of has not. I think it's normal for stuff to progress slowly. Even as we mature as the ecosystem matures, things will start progressing but it's still slowly, right? I'm here to talk about it. I'm a mountain monster vendor and I work with Ethereum Foundation with security staff regarding Ethereum and not so much smart contract development through auditing but more the entire ecosystem and the entire infrastructure of Ethereum and so last month I thought the big change in security scene was that the security awareness and the entire security community around Ethereum had really started to grow. There's been this ETH security group which really become large and there have been a lot of new companies formed and building new cool tools for analysis of smart contracts and this last year the evolution has kind of continued on a path and the tooling has become better and there is more tooling and I think that the maybe biggest change in the last year is that all these security folks who started coming to the scene through smart contracts then also started looking into the base there and looking into actually how the EVM works, how protocol works and they've given inputs to the hard forks and the potential changes in the EVM and things like that. So that's a good development I think. Hi, I'm Yohei, I work for Monstown, I'm based in Tokyo. I bring audits and also kind of the life-changing security consulting for more enterprise-focused solutions. Over the last year I think the manual audit process hasn't changed that much. The tools are getting better but that's still kind of a bottleneck to how we can kind of transition from manual audits to fully automated and I don't think it's going to happen for a couple of years or maybe a couple of decades. It's just a very hard question. In terms of the stuff we've seen this year, I think a lot of stuff related to gas costs. Obviously there was the concept of the hard fork which was kind of delayed because of some changes related to kind of gas pricing. There's been a lot of stuff that we've learned as a company. One thing that we're kind of focusing on is how we mitigate risks even after they happen. So as a security company we're aware that nothing's perfect and all our clients and everyone should be aware that even if you get a security audit, nothing's going to be perfect. So people are I guess becoming more aware of that and trying to plan for what happens if actually a known vulnerability comes up or an involuntary vulnerability comes up. We've been kind of fixing our kind of rethinking through our audit process and how to kind of learn from our mistakes. So I think that's been kind of a good trend. There's been a couple of pretty big security vulnerability findings in the space here thanks to kind of like the security groups. And I think just as an industry in general we're kind of being more proactive and kind of responding more rapidly to kind of these vulnerability findings. I think that's good and we'll continue to progress. Hi, I'm Aleppo. I'm with Open Disabling. We do the tools that people know as form contracts and we also do security audits and we do security research team. And I agree on the tools thing. I think there's been a move forward, a push for more automated tools and these have been developed a lot recently. I also think that there's a bit of over reliance sometimes in tools and what you just said that it's going to be a long time still reliant on them fully. One other thing maybe that we saw is a move not only to a lower level but also to a higher level like incentives and for instance we had the Compa audit this year which created a lot of controversy because we found that actually they could like withdraw entire money so there was a question like should we report these kind of things? Often times we work with no clear threat models so it's like you have to be conservative or the threat model and like in what we report over for our own so I think it's, and sometimes like it has gone past we look at the incentive systems for some reason but we think it's super important and so we do it anyway so I think that's an interesting direction which it has developed a little and we see that we want to see a lot of development for it. Hi, my name is Subin. I work for chain security also doing audits and as Martin said I've been looking a bit more into low level security also this year and yeah I totally agree with your last point that you just mentioned so I'm going to pick it up there so we also see a lot of fights with people over trust models with clients on trust models and so on so like that's always a place where there's tension because the client never thinks of themselves as malicious so that's of course what decentralized can bring to us so that's very interesting. Otherwise in general in terms of the year I would say it's been an interesting year, I would say last year Defconn OpenSephyn released or really announced Defconn OS and I think we really felt the impact. We are seeing a lot more delicate calls than we were seeing one year ago due to all of the proxies and so on so that definitely has happened and still of course we're seeing a lot of bugs but I think we're seeing bugs on a different level now and at the Lightning Talks there was an interesting graph how now the average call depth of contracts has gone up a lot like we're seeing contracts that have call depths up to ten deep or something like some contract that calls Kyber Network then later calls Compound and so on like the complexity is going up a lot so I think security has generally gone up because complexity has also gone up it has still sufficiently many bugs so I would say Germany it's been a positive year. Yeah so with the whole call depth thing that you mentioned I think what we realized there is that a year ago a lot of us were mainly auditing ICO contracts and token contracts and now people are actually building those products and I think some of you here have probably retained some of the people in the ICOs that you were auditing and are now actually auditing the products which they're shipping do you think that they've learned from the things you were teaching them a year ago and the general maturity of their code bases is going up or do you think we're still making the same mistakes? I definitely think that they've learned yeah so I think that I mean for example thanks to the great work by OpenSatman that a certain standard has been set and like it's become a lot easier to write good code and there's kind of like there has been like kind of a template for good code now and that's why I think in general it hasn't moved. Yeah I agree and just like to that for instance we could have like a guide to a checklist for what we expect from the code before starting an audit and I think this has worked and so we still didn't see a code in a much more material stage that we used to see before. What are some of the things you throw in the checklist what do you expect from your clients? Documentation, we put a lot of emphasis in the documentation testing things that talk about the health of the code for instance and we had the solidity and compiler audit last year and a lot of the issues we found were like the bus factor there were two people working on like a language the bus factor is like a factor or like how many people have to be in a bus that rushes for the project to end in time and it was like two guys running this whole industry essentially so this kind of things we now have in this checklist and I think the purchase is doing like a much better job at assessing those. I think the bus factor is probably something interesting for you as well from like what things that happened in Shanghai when everyone was on the same flight and no one was able to respond to things but as the EF kind of like developed new strategies to ensure that if things are found domain that there is always someone who's able to respond or are we still in a two man, three man point of failure there? So there is no like structured way that it's ensured that there are always people on board but as all of it is pretty spread out and our developers are pretty spread out but if an earthquake was to take out Europe then yeah, it would probably be pretty bad for the man. But there are still some people in the USA who cannot vote so there are probably some candidates in that case. Well, I'll speak a little bit to the maturity topic. We've seen people write, better go, people sheet products that we audited their ICOs before or not and I think it's also due to the bear market to be honest everybody was in a, there was a clear look like nothing was happening and people were actually building and it was a meme but it was also true like people were building products and now they're sheeting those products and we see maturity, we see a bigger call that that shows in the code quality that reaches us as a company the audits that we, the clients that we would engage with like they have good teams and like that shows in the code. I kind of want to talk about templating. Obviously Open Zeppelin and all the work around templating has been a major contribution and I think it's a great path to continue down. At the same time as the contracts become more complex there are these custom, there's these features that become more general that people want to use but might not be available on Open Zeppelin so at the same time we've been seeing a lot more copy and pastes of these very complex functions and then when a vulnerability is found in one of these and you just find these cross-contracts so being able to kind of maybe, I don't know, submit these kind of complex contracts for more audits to happen and kind of turning them into templates might be one way to move forward but I mean it sucks that the first company to actually develop it has to kind of carry on the burden of paying for the auditing and the people who just copy it are basically free writers. A lot of people like to use templates but not contribute in any way to it. Yeah, not just templates but like other people's code because some contracts are kind of open source so people just take what's good and move what's working and then they don't have to pay for the security afterwards. So I mean that's a challenging problem to solve kind of distributing the costs across the first person's market and everyone who just copies the code but that might be something to kind of think about. Okay, so now that we've kind of moved away as said from ICOs, from auditing simple token contracts to more complex systems how has that affected your practices? Are you guys finding yourself using tools more in addition to doing manual audits or are you focusing more on doing manual audits or what's changed in your workflow over the past due to the changes we're seeing? I think having a solid understanding of the business model and what they're trying to achieve has been a crucial part of the audit process. Just understanding the business logic and figuring out how that's translated into code has obviously consumed a lot more of our time. If it's like a simple token contract or an ICO contract then it's fairly straightforward what kind of features are available but once the contracts start talking to other contracts and other projects then it becomes a lot more harder to kind of map out what does what and that scope gets a lot bigger. So I think in terms of the stuff we did last year now the kind of unwrapping of the project has consumed a lot more time than it did last year. I can touch on a topic that we were asking Martin before and say that we, like I personally found out that we were, or that we were not that prepared for instant response, honestly. Like as people have shipped more products more stuff has happened like in real time, right? There's a lot more and the groups that were in the security groups that were in like this like people are always discovering stuff and like incident response has become more and more important and I've honestly found out that we're not that prepared that we need really, really, really fucking needed the step of our game. I think for you guys we're also at the security chat. We've noticed that there's people now like shout out to Sam, whoever he is finding bugs every three days in like major all the peace here, like you're amazing. I don't know how you do it. You need to sleep a little more. But I guess with the incident response we can learn from two companies here who've done audits and all of us miss things sometimes for zero X for consensus diligence that was the zero X audit and for chain security it was DAO stack where both of them, I don't remember the DAO stack vulnerability the consensus diligence one missed a a return size check in the zero X contracts thankfully none of the trades were affected but when things like these are discovered what have you learned from that how has that affected your practices what will you do further on to try and minimize that? Yeah, that's also a very good and comfortable question we learned something so that's good and mostly it was processes it's not like Sam is probably an AI I don't know how we got that that's good that's good that he's a good guy and not just exploiting everything from that but internally we discovered that we needed to again we needed to up our game like in processes process wise and mainly in a very specific part of an audit which is mitigations so what happened like for this reason in the static call it was like a problem with the call data return and we discovered that what happened was that we were not treating the mitigation space like the part after the audit after we provide a report after we provide the artifact to the client they obviously fixed the bluff that we find the vulnerabilities now we were probably not treating these mitigation space as a new audit which it should be because like it's new codes so it's a completely different thing that you gotta check I guess and it's a new audit it's just not fixed it's right it's not like new code is new code that's it so that's mostly the most important thing that came out of that that we learned internally is that new code is new code there's no two different ways to treat it there's no mitigation you cannot care less that's just because it's after the audit that comes after the audit that's finished basically so we are now just realizing that we are like we all collectively have been capable of just like to like share these teachings and learnings honestly so we've learned a lot from all the other guys in the community as well so there's been quite a lot of interest during the last three years from my part and what I can say about those is that they've all been different and they can't really we can't apply a template okay so we have an incident to do gather in this channel and communicate with these people because they're all comparing so widely like for example if we can't discover that there's a vulnerability in the Solidity compiler then that's kind of how you privilege information and first a couple of poor people need to find out exactly how what triggers this bug to appear and kind of scan the the entire office of the contract and see what are affected and how bad is it that it actually is going there and can reach out to them approach it by the way it's a really good idea to put some kind of contact information in your verified source code and then we have these other kind of instance where there's vulnerability in GAP which is just a GAP theme where there's like apparently have a number of wallets which anyone can take ownership on where there actually needs to be a group of people who are willing to basically exploit these which can be seen as kind of invasive and and in that part this white hat group has done a lot of good for the community in general the question was also very yeah so a very good question so for us it actually was what was mentioned before trust model so the trust model wasn't entirely clear which contract came for whether that contract can be replaced with an untrustable contract and that's more cost and so yeah it's already well extended it's definitely a matter of processes making sure that trust model is clear making sure documentation is clear making sure that business model is very clear which contract can be replaced with untrustable code which contracts cannot be replaced with untrusted code and then yeah it was a very subtle bug in the end as with every bug when you see it then you're like how could we have missed this but there are just two repetitive calls to a gator which then which is the constant but then of course you can limit it as another constant any more than everything breaks yeah I think a general lesson too is we are all leaders we will all miss bugs eventually so what another means with that another is not a guarantee of security but just an assessment of security we will all miss bugs eventually and yeah so I think this is an important message too that it's important that these things are visible that we acknowledge them when they happen and so that people know what to expect and I don't think that they're absolutely safe because of the paradigm and security I'm going back also to your previous question and I think too deep on the the things we do have on the third ball I was going to say something but I forgot so sorry so speaking about how audits aren't actually a guarantee of security it might be interesting to hear from you guys what is an audit actually and who are they written for so with crypto it's kind of with this industry which we're working with it's kind of interesting where we open source a lot of our audits and that feel like Martin coming from a more traditional thing before Ethereum how did that work there what's different here and do you think those differences are good for the user no matter like should users have to care about a company having gotten an audit is it really that important who's really the client of one of these audits right yeah and I think it ties into what you guys were talking about earlier so to give some context from the regular security industry it's very clear who is the customer or the audit it's the company who wants to be audited and to be you go in and you do a security audit and it's highly classified and you deliver it to the customer and the customer there might be probably only like five people who read it and they probably don't spread it internally because they kind of want to forget all about it but they want to fix it and then later on you confirm that it was fixed and it's a dream of like publishing it and then in this open source world it's kind of expected that these audit results are published and in some cases there's even kind of an expectation that the audit should be phrased in a way that really describes how good this thing is I haven't done audits for the last couple of years I mean in smart contracts but before that I noticed several customers who like wanted to pick out some snippets of the audit where it was really positive work and that's not how I ever wrote the audit so I was put into the audit that this only highlights all the bad stuff I've had that experience where I've written and noted and people came back to me to say that they were sad about the audit result because it didn't highlight the things which they did good so it was like I don't care about what you did good and I care about what you did bad and that's the only thing you pay me to highlight I'm not going to give you two thumbs up on like wow your C20 was really beautifully written exactly so I think and I think that's a good thing to smart contracts even to the open source world but it's better not like Pat's on the back saying you did good it highlights you suck here and here and here and that's it I forgot what the press was I don't know if it's going to take a question do you want to take a question? I have a question so how do you guys identify the vulnerability so how you can make decision that this piece of logic is definitely bad the security part and this is not yeah I guess the question was how we identify the vulnerability I would phrase it quite concisely I mean as has been said at the beginning you need specification and if this matches the specification it's a bad probably also a lot of experience I think a lot of us have looked at code over excessive amounts of hours and at one point you start to get a feel for which lines start to look dingy and then you start analyzing those lines and then at one point you're like yes this is definitely dingy but it's probably for most of us it's a multi or multi-stage process it's definitely about expectations right the mismatch of expectations of the specification then we will try to match the code to that specification like it's about expectations like your expectations if you don't provide us with the specification we will have to come up with one ourselves and this is what Dean was talking about we look at code a lot so we try to mainly just like we try to come up with a specification that we think you want but that might not exactly be so yeah just write your specifications so we would know what to check for otherwise it's really really hard for us to do the job that you want us to and by specification you mean the white paper I need documentation of any sort I can talk about from one part there's this high level thing where you have these specifications maybe written in like flow or business use case kind of thing but then what Dean was referring to I think is more like the lower level assumption you can look at just a snip.code and see what are the assumptions being made here here's an assumption that this value will not be negative and here's an assumption that this loop will end on before a certain time I ask you because I'm also to secure the audits and sometimes that happens that that contract is without any bugs so it's beautiful it's written very good quality code but it works different than something that just write in the white paper so is this a bug or not yes so here's how my process used to work when I was still auditing what I would do is I would expect a specification from my client and then I would go off and look at the code and write a specification for myself and then while looking at that code I look at general bugs and everything and I write my specification and then I compare it to the specification which Dave handed me and I try to analyze do the behaviors which I have found interact with what they've given me and the reason why I write my own specification before reading theirs is because I don't want to cloud my judgment by their predefined assumptions so I go in completely blindly I do that I look at that and then I go off of that and I continue off of that so I think for everyone it's probably a multi-phase process which starts off by just first it's the quick scan for the stupid bugs like stupid calculation re-entrancy stuff like that and then you continue you always go one layer further just to quickly follow up I think that really ties into your question before who's the audience of the audit because actually what we started to do is the whole ICO hype that you mentioned when we started doing a lot of audits was we always clearly wrote what the token does or what the ICO does like for example how many percent of the tokens does the creator get because that was sometimes not so clear on the website let's say and we found this was really important for the people and also if we don't write it in the report even though it might clearly be said on the website now it might not be on the website anymore tomorrow and then people will say like oh but you said this is correct as is and then so we found it's important to document certain core features because to us the general public is also like an audience of the audit report I think it's stuff I agree with everyone but one thing that I think auditors talk about is the centralization issue and how sometimes it's not documented in the white paper and if that's something we should highlight on these reports obviously I think the reports are kind of targeted towards the community in some way but not it's very technical so it's very hard for someone who doesn't understand solidity to kind of go through the report and see how good the code is obviously we're pointing out where the mistakes are so it is going to sound negative but it doesn't mean that the code is absolutely trash even good projects have vulnerabilities that might have an address too and we phrased it that way so there wasn't vulnerability we fixed it but readers might think that oh it had a vulnerability in the first place it might not be a good code so kind of managing this vacations is kind of also important we try to do that we try to make it fair so we point out the mistakes but we also address how they fixed it and obviously kind of like the centralization and also kind of upgradability so the thing with smart contracts is that it's not supposed to be upgradable but there are ways to kind of implement upgradability so in that case the code might be safe for now when the audit was conducted but we can't actually guarantee that in the future so we tend to kind of talk about that kind of work with the clients figure out how to phrase that what they expect and how they can mitigate vulnerabilities that might come across from these upgradability and centralization issues one thing we started doing recently and actually in consensus we were doing it before us and we would like to make a summary post aside from the full report which is less technical and putting a summary of how the project works and things like this bread model we either have right by the client or we assume and a summary of the main vulnerabilities we found and then leading to the actual report where that's the technical so that's a much more digestible thing and thanks guys for coming up with that because I think that's very useful and it's a good way to inform the community without having the technical high bar of reading a full audit report yeah in regard to question earlier which I forgot to actually answer so I think it belongs to good form and should be included in the security audit is basically the trust assumptions of a contract so there is an admin who can do whatever he wants and there are these kinds of admins which can do these kinds of actions so if the report is published then that should be published with it yeah because it should be communicated to the community and I think it is the responsibility of the auditors to outline the trust model I do have a question for you guys so when you talk about the trust model do you highlight that even the specification of the client says that that's what he wants do you feel that there is like an ethical obligation to auditors somehow is that like where you guys are arriving at is there like an ethical obligation to flag that even if it's the specification that's the way they want it yeah I think so because it also yeah highlights that this is how it works these are the built in costs I think there is a question over there so I just want to go back to the public reporting it might sound a bit cynical but there is I think an incentive to disclose every bug obviously on behalf of the auditor especially since these reports are typically a good PR to fund a lot of bugs and they are very severe so I think there is an incentive to exaggerate the severity of the bug or maybe explain it in a way in the report that makes it seem like it was more difficult to find or anything and also if that incentive does exist how do you manage your relationship with clients so they don't proceed this I think the easiest method is not having any marketing people on the team making sure the reports were written by technical people yeah but actually that's not a new problem this is the same thing from classic security and what typically happens is that you deliver a report communication with the client and they say hey that's not really high because there is only like blah blah you can only access the something that's not important and then you can say oh ok I might never put it out to medium so this is and then you deliver a final report that's usually what happens but those reports were public right in the classic world no they aren't public but I mean it's basically the same back and forth you agree with your clients and at the end of the day it's up to the auditors to deliver a report and it's up to the client to decide do we want to publish this or not and if we publish it we can make no changes to it but we can say hey we think that the report actually sucks because we don't think it's that bad yeah going back to the severity classification we try to be as objective as possible like if someone else who wasn't involved in a different process looks at it how superior what they think it is so we do have I think every company has kind of standards on what they classify as high, medium, low informational at the end of the day it kind of depends on how much impact it might have on the end user the people who use the smart contracts so there's going to be significant financial loss and obviously it's high severity but again we do work with the clients to figure out if this will be exploited what would be the impact and how easy it would be to kind of exploit it we don't take it into account like how far it was for us to discover it I think it's more about the impact on what happens to the users so it is a battle we do upgrading and kind of downgrading severity during the other process is common and it really we try to be as objective as possible where are the mics I think there's a question here I think I can, my voice carries two quick questions one is around the tooling and in relation to that with like vulnerabilities that appear only once the contract has been running for a long time so what what type of tooling it's like with acts and diligence what other tools are for you guys using for the security analysis on the one hand and then the second question is around the crypto economic security analysis do you actually do that as well because you have behavior and contract that for example if you have likeness assumptions around staking mechanisms do you and then if it's not fully specified or there's something that's missing then you can't know that it's actually something that there's a plug because they don't check for example for block number I'll say that it's also part of like the conversation with the client not the tooling part, I love that plug yes me that's this great your tools are great there's a lot of great tools in the space though I agree that EVM is also getting open source this Thursday that's an example of a very good at the end yeah sorry not the EVM yet but I don't know how to pronounce it yeah there's a lot of great tools or tools are good but yeah answering the other question about the great economics sense about it I think that comes with a lot with the client normally you'd ask you can see from their specification what parts they want you to focus on but that also comes with that we actually ask the client what do you want us to focus on like what parts maybe it's just a code, maybe it has nothing to do with the economic part maybe they just want us to model the economics behind their smart contracts I think it varies a lot basically so one interesting aspect about crypto economic security which I think is largely overlooked is the fact that transactions float around on the network for a while where they're actually included and people can make bets against what they see is going to happen and investigate the transaction to know what's going to happen and of course do inject transactions before certain events until Diane is giving a talk later today about other bad things that can happen like for example it can be profitable to reorganize part of the chain and move around with how events play out in a way that even though it costs you a lot from the mining perspective the actual value you can cash out from that attack makes it feasible I'm curious to hear about that I think that plays into the fact that we're now building all these protocols and there's all these externalities that in classic security audit probably doesn't need to look at you being one of the main people at EF security who knows a lot of these things on how the protocol works do you find yourself getting more questions from people building things on top of it or do you think you'd like more interaction with people building things on top of the EVM and on top of Ethereum in general to come and ask about these externalities to learn about all the things that can go wrong outside of just the isolated smart contract and yes I definitely think that I mean as I said earlier a lot of the security folks who came into this place to study smart contracts have gradually turned their eyes towards the rest of the infrastructure and that's really great and I think it would be a good thing if more people in that developers and people who wrote smart contracts know more about the infrastructure as I hope Yes I think to that as a plug to anyone working on smart contracts and stuff there's this great telegram channel with a bunch of us in it a bunch of security guys a bunch of adaptable developers called Ed Security and it's easy to join and you can always get opinions from multiple people ranging from not just Martin but trailer bits is in there we're all in there so see if something looks a little weird you can usually go in there and ask and someone will be happy to help and explain from top to bottom what could go wrong so even before and on you can probably go in there and check that out Just a question about that is it a good idea to have anyone be able to join obviously there are some sensitive stuff that are being disclosed there say if like a black hacker joins it's all the stuff that's being talked there I'm not sure what the best approach is to sell that I think it's already above 200 people in it so it's all out there so interestingly with that entire black hat philosophy trailer bits is usually very sensitive on how much of the audit they release because they're from a very traditional security standpoint where they believe it shouldn't directly be open sourced whereas when I was auditing I had the philosophy of if the client doesn't want me to open source an audit I won't even work on it I think in this space because everyone can look at your stuff already if there's a bug someone will have probably already found it especially as we start growing I believe that if there's a bug someone will have found it already they may have just not exploited it yet so if it gets out there's things on like on protocol level as well where there was the Robston hard fork where we had to very slowly it was Robston or Rigby I don't remember Robston where Parity had enabled a feature that wasn't supposed to be enabled yet and there was a network split there was also that problem of how do you start to slowly communicate to all the actors in the network to start updating their nodes on live because it was something that could have been on live and I don't think there's any way around that that person could have easily done it on Robston and then done it on Mainnet and we didn't know who the attacker was to make that transaction and then we actually found out but there was some researchers who were trying out some Casper contract things it was totally innocent but it could have been something completely different you try to keep it silent but I think the main point is acting as fast as possible and I remember being on that and I was quite hectic we acted as fast as possible from contacting my crypto first to update their nodes to slowly get into miners and then get into exchanges and making sure that you keep things vague enough while you can but just ensure that people are actually doing their stuff that it's not like yeah I'll update my node tomorrow I think all the cases were just starting now are cases where the thing is live so I think it makes a lot of difference if it's live a protocol or smart contracts or if it's something to be released and in that case maybe it's much easier to publish the report like open it up for the community because there's no actual risk yet You guys heard what the Joker did in 1989 Batman movie this plot was about not teaming any one chemical but making a series he was a chemical genius and he had a bunch of different products that all essentially were saved by themselves but as they came together sent you into a neurological death spiral so I think about composability with this model that we're talking about now audits that happen on an atomic level they can pass but then people are bringing these money legos and these blocks in such a way that there are vectors that may not have been considered because use cases weren't anticipated and right now it seems like there's not obviously anybody that's looking at it you know when people are putting these things together in an open source way and I'm wondering if this is something that concerns you you see people building on top of audited products and assuming that because they're safe or they're audited in their atomic way that perhaps they're safe in a money legos tower yeah and I think that's a good question and it actually comes very much back to trust assumptions which have been mentioned a couple of times so I totally agree with what Martin said before they have to be clearly mentioned in the audit report and if you see trust assumptions in the audit report that's been luckily brought this up this year which they've come up with and made a really good point there which I think really has a lot to our whole community and so you see the trust assumptions so I think I should kind of answer this question on two levels I think on the smart contract level first of all if you assume that it could interact with anything else then of course compatibility will not always be there but should be there and if you clearly document the trust assumptions then people can see how it should compose and how it shouldn't and on the other side there's also there's also the law level protocol which where I also find this interesting I mean we now have half of coming up with six EIPs and then another one with let's say six or eight I guess entirely clear but there I find it even more interesting because you clearly have this compatibility problem because now we have cases where one EIP influences the other and by themselves they're fine but then you put them all together and they're not so fine anymore so I think because these are closely connected they're composable piece not so clear for smart contracts generally find it a bit easier to think about composability you get that in the traditional info sex space as well a lot of the time you find a tax on like servers and find backdoors it's not actually anyone's software that's a problem it's like a configuration fuck up between two things that leads to that problem so I think we all need to get better at investigating use cases and testing against those various use cases to try and find those problems and that's probably one place where tools can definitely help out where you it's just there's a big world out there and we can't look at everything yeah I just want to draw that this happens at all levels like for instance in our open seven contracts we have like a system of inheritance where you build the things out of smaller building blocks and you have like a very explosion of possibilities and of course you cannot audit all these possibilities and that's like for instance if you do multiple inheritance you have a linearization problem and if you do it wrong you might have a problem and so this is not only like assistance could work on legs but already in the building blocks one has to pay attention I'll just say something really little big we've seen that happen we've seen that happen with ERC777 for example just recently we see that that gets like some re-enterancy problems and we do that every day we do that every day with protocol using the protocol is right and again to that earlier point about 0x the static call we assumed like our trust model assumed that that was okay and it was not like really overseeing the thing that was happening below in the stack even not only about smart contracts but about like the way the protocol plays with those smart contracts there's a lot of problems there that's a hard part about our job I kind of want to bring up the financial side as well if you're a product building on other products or interacting with other smart contracts and other projects that build if you're extremely paranoid about security then you might hate to get all the other things all your dependencies automated as well but that's probably not going to happen most projects do want to kind of control costs as well so again there is kind of this financial limitation as well I think in some sort of way and that's where all the the trust assumptions come in I can't conduct an audit but there is part of the code based on the trust assumptions between other projects and other contracts Hi two things first how valuable are quick informal audits in your estimation if you're in an upgradeable world where you're going to be pushing out a lot of changes and whatnot some are valuable at least to have someone looking over your code just basically giving it a quick read through where you think that most of the value comes from being able to do a deep dive and then bonus question what would change about the the end of the world throw it away to that quick audit thing every code review I've gotten from Nick Jomson or like Martin Hulspand has been more valuable than any audit report I've ever received or anything I've built no offense to any of the guys here because they're all super confident but a lot of the times informal I find informally they also don't care about language and how they talk to me when you're in the business of auditing you have to keep your client happy otherwise they won't come back to you for a later audit when I ask someone like Nick to review something you'll break it in every way possible because it's fun which you can't do in an audit there's a certain limit on what you can do and how would you be to continue having a client I was going to say to ask the guys to do this for you we just did that again quick audit for them was it useful for you? yeah I think it's great but I would say it's the same thing it's getting a second pair of eyes you've made assumptions on and here's your question look at it, you get value I think an important thing is to be here on the expectations you get a quick look if you're here with that I think it can be very helpful I think it also differs on the stage that your product is at you might get a lot from a week in their audit but at some point you should get a real audit so not one pair of eyes like a lot on your code at some point yeah just to quickly add I think it can really help at an early stage especially with the architecture because you can design smart products and how they interact in many different ways and before you visit all the details if you design them in a not optimal way there's a lot of time that we have had some cases like this where you say early why don't you do it like this say to yourself a lot of time and I'll ask a lot of time because it's also going to be a lot easier to audit so regarding what would change on the EVM just minor things so right now when we do a call and we get return value call fails that is zero error is zero and success is one and we know from the Unix world that's just wrong I mean zero that's when it works and then you have an infinite number of ways that can go wrong and now we can change that we can't have it went out of gas or no it didn't go out of gas it actually reverted or some new interesting error happened because of that I need to say something long ago and we should also have a call with value so maybe to you especially because we're I think at F2.0 is like the big buzz with things like that with how the EVM designed do you find yourself talking more closely to the teams working on ETH 2.0 to ensure that we don't fall into these problems like the one thing which I know annoyed me at times is the way storage works in Ethereum is that it's very mathematically sound and every computer theory book will tell you to build it the way the EVM did it but in practice it's never actually done like that and I know Nick Johnson hates this as well it's the word size and that's like something I would change do you find yourself working more closely with teams to ensure that the protocol follows more of these each old design philosophies and is more sound in that design no actually would you like to yeah to some degree but so at this point ETH 2.0 has these different execution environments and I think the vision is that one of them will be EWASM based which is designed already by some other people which is smart and I assume it's good and rolling in ETH 1.0 means we can't modify ETH 1.0 too much and I don't think there's an intention to have like a new kind of ETH 1.0 virtual machine in ETH 2.0 where we just yeah refactor first and I don't think there's a push for that it would be nice if these things weren't so permanent so going back into protocol EIP 1283 delayed Constantinople I think you guys found that issue a bunch of us were on the call discussing that on whether we should delay it or not and EIPs is kind of this like rough thing everyone can kind of submit it and no one seems to review it properly and no one seems to audit it how do we improve that how to ensure that like we don't delay it hard for it's last minute every time because last minute is then when it's supposed to be interesting for everyone to look at how do we ensure that we get into this earlier and don't even get so far as to having to be on the call at 2 o'clock in the morning deciding whether to delay Constantinople or not so Hudson mentioned this briefly in his poll that we should do EEP Central Forks so that there is we change a bit how the process for EEPs could look previously we had someone submitted an EEP and a couple of people looked at it and said like yeah this is great this is going into whatever next hard work is and we said yeah we'll include these five EEPs into that work work which is going to happen on June 1st and then all the work started and problems and problems and as a post for work but instead of doing that the idea is to have the old quarter says yeah these five EEPs look great go ahead with these don't set a date don't set a fork none or nothing and then the work starts they actually build things and they get it accepted into clients once it's been accepted into clients in the form that it can be enabled by testing or by a genesis switches that makes it possible to actually run test cases for it as well so then they go ahead and implement test cases and they do security coverage and look into it from a security perspective and when they tick all the boxes yes it's imperative yeah Python test cases with a happy parent test cases for the work edge cases we've done a security review and yeah from another work edge cases it goes as well then they get back to the work order and say hey we have checked all the boxes when can we roll this out and the work order says yeah how long two months from now let's set fork number blah blah and that's when it's activated so it's not like we have a big heart for coming it's more like this EEP is now ready to just be enabled the work is done more than that incremental staffing I want to give a big shout out to Martin in the back because he was the one asking for security considerations in the IPs so that's a big big big step towards not letting these IPs fork up the network and it's baffling how this is not done before right it's not like it's not really passing a lot of the first like if everybody's just reviewing these things after as opposed to the one proposing them being aware of like how these things can affect the network at least they can give some focus to the security guys right if you are the one proposing it you should at least at least advise kind of trust model say oh this is how this could affect the network so yeah so I think 12 years security considerations are mentioned it's not merged no okay so we should work with that well with 1283 it was interesting because it's like a it's a behavior from the gas stipend which isn't easily found you actually have to know the protocol quite well to understand what the side effect what the security consideration is there which an EIP editor might not even know like the idea is fun but up until you know that nice behavior of how the gas stipend works you can't provide any useful information so is there an effort or should there be an effort where the protocol is documented better with all these like edge cases that got merged in previous EIPs that might affect other things so that when someone wants to make an upgrade to the protocol they can go there and see what they might interact with and if not is that something which we should ensure for ETH 2.0 that like really everything is properly documented because google Ethereum gas stipend and you won't find anything yeah just to follow up on the last part of the question so when I first came into Ethereum one of the things I liked the most and I think a lot of people would actually disagree with me here it was the yellow paper people hate it typically but coming out of the Bitcoin world where code was the documentation this was fantastic like this was way better than was there before and I find that a bit sad well again I understand it has been like I don't intend to blame anyone but it has fallen a little bit behind right so like not even the constant denormal codes are part of the current yellow paper so like you don't have all the current op codes in the yellow paper so that's of course not ideal I mean I guess I could also say another word in another way saying like a big shout out to like Yoji and so on who did a lot of the great work back then on the yellow paper so unfortunately we haven't been able to keep this up and I think there we have to do a bit more again because it was to me a lot of people like to flame but like in the end there's one place where you can go and you can check that has always been super helpful to me so I think that would be great and so and to quickly to a previous question about the EFPS on it's actually it was mentioned before it's a composability problem right like so because like you have these low level problems of like gas costs and so on and you have high level things like solidity brings things a certain way and so on and transfers and so on so when these come together then there's these assumptions they don't really mention anymore and it's also yeah what we saw now with this new hard work that there were some things being broken where people made assumptions about gas costs being fixed right and now cost of gas load is going up which is a good thing but like still it's breaking some things that people were not expected to be broken ever and so on so it's becoming more and more clear that actually there should be somehow more connection between low level and high level at least there should be more discussion about it it should still be clearly separated because the developers don't want to think about it and they shouldn't have too much but there has to be some more interaction between these two worlds I think so composability is the problem because that means Polkadot is going to have a lot of fun I think we're getting really close to an end here there's a question yeah sure over there I have two already questions so given that nobody can guarantee there is a smart contract so the first question is do you feel that in bitcoin there's a time lock type of mechanism allowing withdrawal hold in a certain period of time so you can call back you think that's a good idea to handle withdrawal the first question since that we are ongoing and we try to anticipate some zero day or maybe responsible disclosure of any issues that you haven't seen it so do you think it's good idea to put some kind of like pure switch either freeze the spam or have a quickly move the funds out with some higher privilege like what we see it's a kind of good best practice in terms of handle this kind of zero day disclosure or kind of like you know ongoing auditing finding that we don't anticipate in the open sibling contracts both the time lock contract and the also functionality chart too so yeah I think both are good idea what we used to suggest is have those in there up until you can prove that your code is mature enough to then feel safe to turn it off I think we're still so early in this space that we can't consider that a backdoor so for instance what we did with compound so what we were talking before is suggesting not only us but the community in general suggesting a time lock where you so if they want to upgrade the system or they want to make a major change they will need to wait for some period they introduce the change and then we have to wait for a period for the change to be implemented so people can if they don't trust the change being implemented with the other money so that's a solution and then of course there's governance and there's many governance solutions so they're from like a basic multi-seq to a DAO or whatever so I think this is like a huge copy but it's a really valid concern another question what advice would you give to someone who would one day love to do what you guys did for security auditing don't it's not that fun yeah a lot of responsibility and none of the fame just kept playing yeah just kept playing for shit like I hear that question a lot there's surprisingly there's a lack of resources basically I think the answer is that there's a lack of resources for security in the Ethereum world still even in the blockchain world but specifically for Ethereum we maintain the smart contract best practices repo which is like it's basically a community effort by now like we still maintain it but it's mostly crowdsourced people just like we just merge changes and keep it updated but it's the best one that I know of and that's kind of we have the Ethernet a game where different levels where you pack your way up launch that like two headphones and now it's kind of by the community and it's in the same world it's super interesting, super fun to play so that's a piece of side of everything learn the protocol, learn the EVM learn how Solidity works, how the compiler works and then start giving your friends free audits or code reviews and slowly at one point they'll offer to pay you that's how at least it works for me there's a couple more CTFs that there's oh man I forgot the name of the Captured Ether, yeah, Captured Ether by Steve Marks which is really really good and really fun and there's a couple more that I like we've also been maintaining one life from at least CC that I'm happy to share after this couple of CTFs, yeah just a photo, to what extent are some audits public to look at audits or previous audits as an educational tool is that a thing? yeah, I was going to recommend the Opens I think audits yeah, I think everyone will know a lot from those any other questions? I have a question about the processes like as what's all the saying for 0x there is when you deliver the report there is a fix that's coming up and you have to deal with it as a new audit and you already have new clients and new audits so how would you juggle between those new audits and the previous, how would you juggle between those new audits and the new mediation, new code and also when is when are you done with the code when you say it's a good code don't take on too many clients so you're doing a lot of fixes to code we already audit or new rounds for all clients so when you the initial report they come back with fixes and they look at the code again and this goes back and forth and then so for that part it's part of the service that clients will come with implementing the fixes we recommend and then who will look so it's part of the fault so it's a different thing and then for the new code of all clients it's maybe we try to prioritize them if we work with them before it's very complex from like classical security we would typically I mean in the follow up you verify that the findings are fixed and if they also re-did the whole thing it doesn't get a new audit and they would have to buy a new audit but this is like only a couple of take the checkboxes just these things that we found were fixed but it's not like that to do audit on the new code base just to follow up so if there is something found in that code is that you're calling to respond? So it's very important that you as an auditor to cover your own ass write down exactly we wanted this hash of the code in the repository the follow up was made on this hash and it specifically only covered checking that the previous findings were fixed and you should put these kinds of text in to specifically say what you did what it is and if you don't do that just you can get blamed later that these guys didn't see this whereas if you have it you can say yeah but that was after this metash and follow up The directory is a fun story with a classic security company on SSH they found a critical buy on SSH and they reported it it got fixed and then when they suggested a fix something else was implemented and they looked at it and in the end there was a vulnerability introduced by the fix that was not the one they recommended and they got blamed for years for introducing a backdoor in SSH so yeah it has to be very careful Partly it was a litigation so the ball around was reporting some memory errors and they took out the zero thing which let you weak But it might be a good tip to look at some classical security reports standard security reports and templates for that and you will see that there is like five or six pages of just boilerplate text saying like what the security what it has covers what it is and what it is not and maybe two pages there were actual contender findings and the rest is just very much get a really good lawyer for the company are there any other questions otherwise we are going to wrap this up because we have pretty much run out of time Thanks guys