 We are late, we can try. Hello. Can you all hear me? Oh, Melasia. Yes, Mr. Kunikun-Golfan, we can all hear you. Can you hear me? Oh, good to go. Okay, excellent. Perfect. Okay, let me just share my screen one sec. And I just need someone to tell me if you can see my slides. Yes or no? Yes. Beautiful. Okay. All right. Well, if it's okay to get started, I'm going to talk today a bit about end-to-end encryption. And by the way, I really enjoyed that last talk. I didn't see the entire thing, but it was really, really interesting and I'm actually curious about the iMessage question. I'm going to talk today about end-to-end encryption and give a very high level, not terribly deep and technical state of the technical and the policy debate about end-to-end encryption. I should introduce myself. Let me actually do that on the next slide. I'm a cryptography professor at Johns Hopkins University. I've been working on this particular issue, end-to-end encryption and sort of the policy and the tech issues around this in the United States obviously for about 10 years. And of course this is a problem that has a huge international component and I'm going to confess right up front that obviously my viewpoint is very United States centric and I only have a very much more limited idea of what's happening overseas. Just, I will qualify that several times. I'm a co-author of a technical and policy document that came out in 2015. I believe it's called Keys Under Dormats and what it really is is kind of a position paper that says here is why you should not include backdoors in end-to-end encryption. And it's a really good document and I will confess that the lion's share of the work was by my co-authors, but my very, very distinguished co-authors who include people like Ron Rivest and Whit Diffie and Susan Landau and so on. I've been involved in a number of things on the edges and I just want to list these. Not all of these are academic work, but they're things that I am happy I got to be part of. So this last year during the pandemic, somebody came to me, it was Alex Stamos came to me and said, we need to fix Zoom. And so this is so relevant because we're here on Zoom. We need to add encryption to Zoom. We need to make Zoom more secure. And so I sat down with this really great team and we built a system to add end-to-end encryption to Zoom. A few years before that I was involved in a project. I heard Apple iMessage mentioned. So I was involved in a project that found vulnerabilities and iMessage encryption. I have great stories of trying to interface with the Apple team about that. At some point, I think it was in 2016, I was invited to Facebook to look over their new end-to-end encryption system called Secret Conversations. And a few years before that, I was very fortunate to be involved in some of the Snowden leaks and the reporting on that, mostly around how encryption was being compromised by the U.S. government. And that was a fascinating time. We learned a lot. I'm pretty sad that we didn't take a lot of durable lessons away from what happened there, but I think the lessons of what governments can do when they want to compromise encryption is something that we need to remember. Good. Okay. So today I'm not going to talk about all those things. I'm mostly going to talk about the history of end-to-end encryption systems and what we call the crypto wars. And the crypto wars is kind of a shorthand, which many of you are familiar with, for how governments have tried to tackle open encryption systems and make them more tractable and more secure for, sorry, more available, more data available to them or to prevent the deployment of cryptography. I'm going to talk about some technical details, including weaknesses and research, open research areas. But mostly this is a policy talk, which is intended for people who have not been following it to be very closely. So those of you who have been following it to be, we'll find it a little repetitive, but hopefully you'll learn something new along the way. And of course, like I said, it's a very US-centric talk. That's just where I live. I spend time talking to policymakers in Washington, DC. I know how they think. I would love to learn more about how people in India think and I would love to know more about how this debate is moving globally. I just don't have as much access to it. So I'm going to go back to the US debate. All right, I'd like to start this talk by going back in time. I want to go back to the 1990s. These folks who you see on these slides, they're pretty well known. Some of them are pretty well known at least. That's Phil Zimmerman, the inventor of PGP, at least back in the 1990s. That's Dorothy Denning on the right. She did a great deal of work around key escrow and how to deal with the clipper chip. The folks in the middle, that's the, this is not actually a picture from the 1990s. This is the open SSL team. Open SSL being a very relevant part of the crypto wars and the development of crypto in the 1990s. So going back in time, back to this 1990s period, was really an interesting time. So if you think back to the 1970s, we got the first public block ciphers like the data encryption standard in the 1980s. Academics began to really develop open public key crypto and do interesting things with it. That was the development of zero knowledge and all sorts of powerful crypto primitives. But it wasn't until the 1990s that public open source software to do useful things with cryptography actually came out. And the most famous of those programs came out in 1991. It was called Pretty Good Privacy or PGP. It was written by Phil Zimmerman. And there's a story behind that of what inspired Phil Zimmerman to write this software. And I'll come back to that in a moment. But what you should know about PGP, if you don't already know about PGP is it was the first piece of PC software that lets you take advantage of public key crypto. The 1991 version was pretty awful, but it was crypto analyzed and improved and became something that was very powerful. There are other things that happened in the 1990s. There were DES challenges. So people started caring more about crypto. In 1994, something very important that's not end to end encryption related. Something very important to happen, is that the first version of the SSL protocol was released by Netscape Communications, which suddenly made encryption into a something the industry required as opposed to something that open source advocates cared about. And in 1996, new crypto libraries were put out and 1998 open SSL came out. That's kind of the history of what was happening in the 1990s. Governments did not handle all of this well, or at least the US government did not handle all of this well. And part of the reason is that historically encryption and cryptography had been a subject for government agencies. It was highly regulated. Exporting cryptography was actually illegal, or at least you required a license to do this and it was very difficult to get one. Now, how can you prevent somebody from exporting cryptography? Cryptography is a bunch of mathematical equations and maybe some software. And of course in the US that's very, very close to speech and we have a First Amendment that says, you know, you can't block this. However, the reason this was so easy to do is because cryptographic knowledge was not very widespread and because a lot of cryptography in the pre-1990s period involved hardware. So when you talk about exporting cryptography, typically you meant exporting some physical device. All of a sudden in the 1990s, exporting cryptography meant exporting a program. And we had a brief, very strange period where the US government tried to regulate the export of web browsers that have SSL capability and you probably remember those times if you were alive then where there was a 40-bit international version of the browser and 128-bit US version of the browser. And the way that you would decide which one you got was your IP address, which was hilariously stupid and of course anybody overseas could still get access to the strong one if they wanted. This led to all sorts of problems that mostly we cleaned up but they lasted for many years beyond this. And this all kind of culminated in a period in the 1990s that we call the first crypto wars. Crypto wars is kind of a colloquial term but basically the idea is in the United States we had these laws and these laws prevented the export of crypto and of course national security agencies wanted to maintain some kind of monopoly on crypto and the ability to crypto analyze people's messages. In the 1990s of course PGP and other software just threw that out the window. It was no longer possible to do. There was this tipping point where suddenly governments had to face up to the fact that people criminals could download encryption software that was more powerful than their technology could break. And so they tried to come up with some kind of compromise very famously in 1993 the US government with the help of the national security agency developed a piece of hardware called the clipper chip. Clipper chip was a key escrow technology that said it was for phone calls that said every time you make a phone call your phone call can be encrypted if you have this chip. But of course most critically the chip would also send a copy of the encryption key along with the message and that copy was encrypted under a master key which was stored split across two US government agencies. And the folks at the NSA thought this was a great solution. Now we can have encryption so businesses could make encrypted phone calls. But of course the government when they needed to could come with a warrants and they could decrypt those calls. It didn't work. The system turned out to be broken for a variety of different reasons. But more importantly the idea that we could replace software with these hardware chips was just not viable. And it turned out that if we had adopted that system it would have pretty much slowed down the development of industrial crypto by many years. And so we pretty much abandoned that mostly for very good reasons even leaving aside the privacy issues. And at the end of the day the result was primarily that we kind of moved on to a new regime. I want to show you this picture. Many of you will recognize this gentleman here on my left the one holding the pen in his hands. This person was a very important figure back then and actually he proposed some legislative language in Senate Bill 266 during this entire debate that basically demanded that telcos and maybe ISPs and so on include technology that would have forced them to reveal the plain text contents of voice data and other communications when appropriately authorized by law. And this was a piece of text that would have actually mandated the inclusion of either key escrow technology or no end to end encryption inside of these inside of any communication technologies. And fortunately for a number of reasons this this bill this language was not passed into law it was removed from the bill but it gives you a sense maybe of where things were back then and maybe where things will go again at least here in the US. Okay so we can sort of report the result back in the 1990s many people said well we won we won the crypto wars and we're not getting the technologists and the pro encryption advocates and the results was that export regulations were not repealed but they were relaxed and open source crypto in particular was given a kind of an exception if you write some piece of software and release it on GitHub you do not need to apply for ITAR or export licenses. National escrow proposals like Clipper were just abandoned but even better our national laws there's a law called the communications oh gosh I can't remember access for law enforcement something called Calia currently does have a provision that says you must turn over encryption keys or decrypt content if it's encrypted in a cellular provider or an ISP or a telcos network however the exception on this is this is only necessary if the provider themselves has access to the keys so in other words the current law as of today in the United States is if somebody uses end-to-end encryption on an ISP or a telco and the telco doesn't have access to the key which they shouldn't then there is no need for them to do anything and this applies to telecommunications companies and potentially to companies like Apple and WhatsApp and so on so there is currently no law that allows the US government to force you to decrypt end-to-end encrypted communications and that's a big deal okay well the great thing about the crypto wars is that quote-unquote we won the bad thing is at the end of the crypto wars which is the late 1990s nobody was using end-to-end encryption and nobody is a relative term some people were using PGP there were key signing parties and so on but the usage was very minimal so I guess the government decided well we can make an exception here because there's no threat nobody is using this technology it doesn't really make our lives harder in the 2000s things got more interesting things began to change and so what happened of course and I think you can sort of look and see what's going to happen based on the group of people that I've chosen here is that technologies began to come online that made encrypted end-to-end encrypted messaging and encryption in general something it was feasible I'm choosing to begin this in 2004 Goldberg and Borisov developed this piece of software not widely used but slightly more widely used that was called off-the-record messaging or OTR and OTR by itself it never reached mass adoption but it ended up being used on instant messaging protocols like AOL, instant messenger and Jabber and I think IRC and I'm citing it here because it really introduced two new concepts that I consider to be kind of the basis of our modern encrypted messaging infrastructure and one was this idea of message deniability that people who are using the system should not have their messages signed so that it's easy for somebody to attribute message content to them cryptographically that's not so terribly important but the other portion of the OTR system was the idea that messaging conversations go on for a long time and so the idea is that when you send a message that message should not, you should not retain the decryption key on your phone to decrypt those past messages and so the idea that was introduced in OTR was this notion of what's called forward secrecy that after you send a message the key, the decryption key on your phone updates in some way so that if somebody steals your phone they can't read or at least they can't decrypt off the wire all of these past messages and this was implemented and this was advised something called ratcheting which later became very important for the protocols that we use today but really the most important thing that happened happened in 2007, I don't need to explain this it was the development of the smart phone smart phones changed everything smart phones took us from a world where we were using desktop computers and IRC and moved us over to a world where we have a portable device with a relatively powerful processor and the ability to hold cryptographic keys and created a world where suddenly we wanted to communicate with people using keyboards and electronically in a portable way and that made messaging go from something that was this use case that only hackers on the internet really cared about with IRC to this mass adoption strategy where suddenly everybody was using SMS and other apps to communicate with each other and again this history is really well known and repeated in 2011 Apple launched a service called iMessage iMessage we already talked about in the previous talk but iMessage was revolutionary from a cryptographic perspective it was revolutionary because it used end-to-end encryption natively and it went out to millions hundreds of millions of people in a single software update the actual cryptography was pretty terrible and it was the same as what we did it uses RSA encryption plus some AES and it uses a centrally hosted server to hand out encryption keys that's operated by Apple lots of problems there that I don't love but forgetting all the details it actually turned out to be broken, we broke it in 2016 but forgetting the details of that it took us from a world where encryption usage was measured to be tens or hundreds of millions and many of those people didn't even know they were using encryption you pick up an iPhone and you see blue bubbles that means you're using iMessage but to the government that's a massive change the second thing that happened over this time period is that two inventors who the name should be pretty familiar Moxie Marlin Spike and Trevor Perron took that OTR protocol and they said let's build an app let's build an app and a new protocol that's an extension of OTR and we'll use that to do mass messaging they renamed that to Axolotl which was an awful name that everybody hated and then eventually they renamed their protocol to the name Signal Protocol and the app that deployed it first was called Signal and probably most of you have that on your phone and then later in really kind of a coup they were able to add the Signal Protocol to WhatsApp somewhere in between 2014-2016 and then two billion users so the change in the number of people using encryption just was dramatic and suddenly this is not something that governments can ignore which brings us to 2010-2013 or so which is what we're starting to call the second crypto wars this is a website that is currently maintained by the FBI it's been here since about 2013 and it's called the Going Dark website Google FBI Going Dark it's a beautiful branded website they spent a lot of time on this it was actually initially a priority of James Comey who was the previous FBI director and essentially what it does is it sets out starting in 2013 that encryption is going to be one of the biggest technical challenges that the FBI has in order to perform its criminal investigation function and it's really ambitious it doesn't say encryption of messaging it doesn't say encryption of phones it says all encryption we need access to encryption bypasses so we can access these devices and these communication systems and without this we will lose capabilities and we will no longer be able to protect the United States against everything and keep in mind that the US FBI is both a national security agency and an organization agency so they have both functions and they care about the ability to break encryption as part of both of their functions they also coined a term that they continue to use which is called warrant proof encryption and they use this on their website warrant proof encryption creates in effect lawless spaces that criminals, terrorists and other bad actors can exploit so none of this should be surprising if you've been following this debate in 2015 things came to a head here in the US where Apple had a phone that was used by a pair of terrorists people who had performed this conducted the shooting the phone probably didn't contain anything they had actually destroyed the phone that probably did contain information but the FBI decided they wanted the back door they wanted some way into this phone Tim Cook said no there was a huge fight it ended in a sort of unsatisfying resolution when some technical folks there was a story in the Washington Post this last week some technical folks found a technical bypass that let them into the phone so the FBI did not get to establish any kind of legal precedent and Apple didn't get to fight them off and establish a legal precedent the issue was just postponed I also should point out that this issue is ongoing this is from 2019 but I don't think this is an issue that has left anybody's priority queue of course many governments all over the world have their the same concerns about being able to access and encrypt messages India appears to have a more unique problem and I've heard this several times which is that not only do they want kind of the same investigative capabilities the ability to access plaintext and so on but they also want to track the forwarding of messages particularly on WhatsApp because of course there have been a number of incidents and so this is a really unique requirement it doesn't come up very much in the US debate but it's very very specific to India and it's a little more tractable but it's something I'm just going to mention talk about a little bit later going back to the United States of course we have seen legislative proposals that try to add these capabilities of being able to decrypt information there was recently one that happened in the middle of last summer where a group of US senators introduced a law more or less what Joe Biden said back in 1991 if you operate a service you would better be able to give us plaintext the building passed into law but it shows that there is a pretty strong interest in continuing this kind of legislation nearly none of this history but I should mention that the five eyes governments including as well as India and Japan have all been combining and joining forces on asking for encryption backdoors and more which I'll talk about in a second so again this is not just an isolated series of governments doing what they do it's all over the world and it's in coordination okay good so now I've said all these things and I want to talk a little bit about what governments are actually asking for because I think it's a little bit complicated and I don't think even governments fully understand what they're requesting so let's be clear when it comes to messaging and I'm going to keep this entire talk on the subject of messaging when it comes to messaging what they're looking for obviously their preference is the best kind of encrypted messaging service is one that doesn't exist one where the messages are not encrypted they're sent in plaintext and they're retained in long-term stores at providers so a warrant police can show up and say give me all the messages sent between these people and the problem is solved that seems to be a very very strong desire or at least if we don't have no encryption we can slow down the deployment of new encryption failing that there is a strong push to develop what are called key escrow systems and of course this is a very technical group of people I think but just to be clear let's define this key escrow means that every time we send a message we're going to have some master encryption key which could be held by the government although the US does not seem interested in holding keys but more likely held by the provider or you know company like Facebook or Apple and that master key can decrypt on demand whatever message needs to be decrypted and that seems to be the general push I just want to stress here how scary that requirement is right so if I have a master key sitting somewhere that can decrypt criminals messages then there's nothing technically to prevent that same master key from decrypting everybody's messages so if that key is stolen everything is clear text to the person who does steal it that's a very strong requirement the more interesting current request that is actually getting a lot of traction I think unfortunately in the United States is actually kind of a very hard shift so governments have mostly been asking for the ability to what they call get exceptional access gain access to messages when they have a warrant which in the exceptional means this is not the normal case we don't want to read your messages every day it will show up with a warranty it will happen very occasionally it will be the exception to the normal message flow however very recently an entire group of governments and particularly here in the U.S. the Attorney General signed on to a new requirement that comes kind of out of nowhere for us and it basically demands that we have real-time content scanning and the idea here of real-time content scanning is that we can look at every single message that you send through the system and we can check it for certain properties and the most common property that's mentioned in the U.S. is this an example of what we call CSAM or child sexual abuse media and it's a very powerful request because none of us I mean I have kids none of us wants this to exist none of us likes the idea that this kind of media is flowing through systems and of course the idea that the transmission of this kind of media is horrifying it brings out an emotional reaction in everybody but from a technical point of view it's really important to understand the difference in the requirement in the ask what governments are asking for between asking for exceptional access and asking for real-time content scanning the first is an exceptional capability that is not used except in occasional very serious legal instances when a court is granted a warrant the real-time content scanning is a ubiquitous workflow every single message is being checked maybe not every message is being flagged if the system works well but every message is being scanned and from a technical point of view that's a very very big difference there are some governments who I think are being a little bit more reasonable in the sense they're asking for more targeted wiretapping and eavesdropping the UK has made a proposal for that that's a little bit more reasonable we'll come back to that in a second one of those is called the ghost users proposal which GCHQ in the UK came out with and of course I mentioned previously traceability this is kind of the new one for me very hard and we'll talk about well potentially hard we'll talk about why that is and then so I don't want to spend a ton of time on Kiesgro because I think most of you really do understand this but I want to present it from the governments point of view which is kind of this is what the governments want they don't care about Kiesgro they don't care about key storage or any of the details they basically have been demanding at least in the US that companies should be able to produce plain text on demand it's interesting because in the 1990s when Clipper was the proposal that system was actually very very carefully developed so that US agencies US government agencies would hold keys the master keys there would be a process to split the keys across multiple agencies and everything was done centrally by the government with certain protections in place the various proposals that have been made recently do not include that protection they basically say hey you figured out industry go off and figure it out which means of course you can imagine Apple which is very well funded could probably build something more reasonable but you can imagine there are a lot of other companies very small companies that are going to be figuring this out from scratch we're going to do it very poorly so this is not necessarily an easy ask so obviously some of those companies will say well we just can't afford to do encryption at all if this is the requirement that's a very likely outcome but generally speaking what it means is that somehow they're going to have to encrypt session keys for messaging using some kind of escrow key that key is going to be stored somewhere typically on a messaging provider's hardware security module and we're not really sure we're going to do that that's a very very hard thing to do and we know this is a hard thing to do because the US government tried to do it once and the result was the clipper chip and it was broken not necessarily in its key escrow function but it failed its design which was created by the US national security agency had a field with a 16-bit message authentication code and Matt blaze at Bell Labs was able to figure out that he could bypass the encryption the key escrow function on this chip relatively easily and partly as a result of that design failure support for this kind of government design key escrow collapse now this is the US national security agency whatever you may think about them they're supposed to be good at designing encryption technology so the fact that they failed on that aspect of the clipper chip was a good indication that maybe this is not such an easy problem to solve so key escrow in the US is not very easy to do and of course we couldn't have done it it would have really been very bad for software security and so on we also had a really interesting example of how bad key escrow could go wrong even if you protect the master keys I like to cite this to government officials at least here in the US because none of them have heard of this but many of you or some of you may have heard that in the 2000s this came out during the Snowden slides the US government created what we believed to be a backdoor pseudo random number generator it was called dual EC DRBG and there's a long story I'm trying to cover very quickly but we believe that this algorithm had a covert backdoor and it was put out in this standards and for reasons we still don't fully understand this algorithm wound up inside of juniper net screen firewalls in about 2008 and they continued to be there through about 2012 and actually 2015 and so on in 2015 a group of non US hackers we believe possibly the current rumor is possibly Chinese state sponsored hackers were able to get into juniper's net screen code base and what they did was really alarming what they did is they took this possibly backdoor algorithm and they replaced a single 32 byte field with a key that they had generated themselves we don't know how they generated but clearly they did this for a reason and what we know about that in fact we did some of the research just to verify this what we know is that this creates an opportunity for anyone who has the corresponding secret key in other words the hackers themselves to decrypt any VPN connection that was made by these firewalls and so essentially they took our backdoor and by our being very specific the possible the NSA is highly likely backdoor and they turned it to their own backdoor and I think this illustrates kind of the risk you know there are so many risks around key escrow systems and duly CDRBG is one this is an example of how dangerous they are and you cannot hold these systems against very sophisticated state sponsored attackers and this kind of shows us one of the possible outcomes I don't want to praise other backdoor approaches but in a few years ago GCHQ in the UK came up with their own approach and I'm not in love with this but at least it gives you an idea of what's possible GCHQ proposed an alternative approach to creating some kind of backdoor into messaging systems and what they identified is really valuable they pointed out that most messaging systems have a server server's job is to when I want to talk to Bob I go to the server and I say give me Bob's public key and this is called the identity subsystem really it's job is to hold on to who's part of the system as well as their public encryption keys and give them to me when I need it and all of the systems that we're familiar with have a server like this Apple iMessage, WhatsApp Signal they all centralized the server to distribute public keys when you go and you try talking to a new user you're trusting that server to do the right thing you're trusting it to be not compromised most systems have some kind of key fingerprints you can verify that but nobody does the idea that GCHQ proposed is well here's the weak point in all these systems let's attack the weak point and what they want to do is they want to subvert that identity management system they want to subvert that server to insert either wrong keys or add new users to make a private conversation to Bob it's not just me and Bob it's me and Bob and maybe the FBI and this seems very simple of course then the question is why doesn't my app tell me that the FBI is present but of course the problem is that you know that they're also proposing that we change the app so it doesn't reveal facts about who's part of the meeting once you start doing that pretty much anything's possible but I will give GCHQ credit at least in this case at least a system that does not have the same vulnerability as the key escrow approach if you steal the keys from GCHQ you won't be able to decrypt anyone's message you'll have to actively target a particular user and that's not always so trivial so this is at least slightly lighter approach even if I think it's unworkable okay I want to briefly talk about tracing and I think what's kind of notable about this part in this idea of tracing different people's past forwards or attachment forwards what's really notable about the problems in tracing is that it's just a problem we don't quite know how to solve so here is an article that talks about an Indian government proposal to do some kind of hash based tracing and I think the idea is basically if somebody is forwarding an attachment and it goes becomes very very popular we can find out who started this or we can go back and see the path by which it arrived the government in India has proposed some kind of hash based tracing I've seen some research that proposes other ways to do this it's possible that what's up already has some way to track forwards it's a little bit difficult to tell but it's not necessarily the case that we know how to do this because the situation is sort of fluid some people have proposed tracing approaches that are privacy preserving in that you can only trace it as if they achieve large scale virality but the problem is we don't know when a message is just starting out if an attachment is sent out from one person and it reaches 10 people and then it goes to 100 people and it goes to a million people at what point does it become traceable is it always traceable from the point where I send it to my 10 friends or is it traceable at the point where it reaches a million and then somehow when it reaches a million people can we go back in time who sent the original 1 to 10 etc viral push and so these problems I'm not saying these problems can't be solved I'm saying we need a lot of research to really figure out what is being asked for if we don't want to just build a system that traces everybody's messages everywhere and this is kind of an example I think of what governments are asking for I'm enthusiastic about solving problems I am uncomfortable with the idea that governments are saying we need a system today when we researchers don't even know we need a system that is just generally one of my concerns so I think this is largely a research problem I believe I have about 10 minutes to leave time for questions so I want to talk about another issue another area that really really concerns me and this is a place where I also believe that we are just at the research stage and governments are demanding that we deploy systems this is an area where I'm actually really deeply concerned about how to make a terrible mistake and this is the area of real-time content scanning I'm pointing you to this is a US Department of Justice press release but actually it's not just the United States it was a joint release that was made with a bunch of Five Eyes countries as well as India that talks about how we need to build real content scanning systems this is signed by William Barr who is the Attorney General under President Trump basically says we need you Facebook to stop deploying encryption systems unless you can build real-time content scanning systems in so that we can prevent sexual exploitation of children and so on and there's a very noble and very good motivation but I want to talk about some of the issues with that again the motivation the US is mostly child abuse sexual imagery or child sexual abuse imagery which I'm going to refer to as CSAM over and over again so CSAM is horrible stuff but there are also other proposals for terrorist imagery and other things like that using the same technology so we won't be too specific about it so let's talk about kind of where we are today and I'm going to come back to the CSAM debate before we get to CSAM and how that's a problem I just want to kind of give a quick update in where we are amongst the technical and policy wonks and these are not just the pro encryption people these are also the national security wonks here in the US as well as other folks from the UK I know have weighed in on this I think that we are approaching a growing consensus that end-to-end encrypted key escrow for messaging specifically is not a great idea I think this is a reluctant agreement because I think that the policy community has started to get the idea that we don't know how to do this safely and that the attackers after these keys are just too powerful and as evidence for this I would give you a report that came out a couple of years ago called the Carnegie Institution report on I'm not sure what the title is end-to-end encryption and how do we deal with the national security debate and what they concluded is that in the setting of physical devices like phones key escrow could work and they argued that specifically in the setting of phones decrypting phones this could work because you have to do surveillance on somebody you first have to get their phone and so the idea of building a key escrow system and allowing for mass surveillance if the key escrow system is compromised it's very hard to do that against phones because first you have to get my phone whereas with messaging systems what they call data in motion the data all you have to do is compromise the right server and you can get access to millions of people's messaging which means that key escrow systems could very rapidly lead to mass surveillance and there is a sort of an agreement a reluctant agreement that this is not a great approach and so as we've noticed a big shift has been to move towards the push towards content scanning as kind of a new ask and so what we're asking for now is real-time ability to scan for CSAM material but of course the implied compromise here is that these systems will be built only to scan for abusive content yes every message you send or an encrypted messaging system will be scanned but only to see if it's child sexual abuse material not to see if it's a criminal message or if it's something that's politically charged and it's also a very good way political way to bring the debate into an issue that people really care about the biggest problem with this area the content scanning area is that we, meaning the technical community has no idea how to build it with end to end encrypted messaging I've had recent calls with folks from Google who said you know we really want to build this we really believe in content scanning we think this is good can you tell us how to do it and nobody knows researchers are still thinking about it and you know doing it there's doing it and there's doing it safely and those are two different issues so this brings me to this list of things that governments are asking for that we just don't know how to do in content scanning which is essentially everything that we're kind of being asked to do so we move from a world where we don't know how to do something just safely to a world where we literally just don't know how to do it at all I'm going to kind of skip past just in the interest of time I'm in the past I think I've said enough about key escrow but I want to briefly finish up for the next five minutes by talking about content tracing and how not content tracing sorry I'm going to skip that content scanning and what the problems are with content scanning I'm going to show you how content scanning works today in an unencrypted system so this is an illustration I've used Facebook as the example when you send a message through Facebook messenger an unencrypted message through Facebook messenger what's happening to that message is it's being sent to Facebook servers in clear text obviously it's probably TLS encrypted but it arrives at the server in clear text Facebook then uses what's called a media hashing algorithm which is a fuzzy hashing algorithm or a fingerprinting algorithm to generate a hash and then what they do is they check that message against a hash corpus and that hash corpus is a list of I think 14 million abusive images that they keep and constantly up to if the message doesn't trigger if it doesn't get a close match against any of those hashes they send it on to the client if it does trigger then a bunch of things happen that person's account may be investigated the image is going to be manually examined by folks at Facebook it's going to go to an organization here in the United States called Nick Mac which is the national center for missing and exploited children and if the sender is a US user they will hopefully be arrested if they're an international user pretty unlikely but that's kind of the process today and of course these media hashing algorithms a very famous one is something called photo DNA and it's a fuzzy hashing algorithm because the idea is that you can't use sha one or MD5 to match these files because people have different encodings of the same file like a JPEG and you know a GIF or a PNG will have a different hash even if they're the same image so these fuzzy hashing algorithms look at the way an image works there is another model that's coming up right now which is essentially a system which uses a neural network Google is starting to deploy this of course we know how neural networks work they can take you can train them on large numbers of images and they will trigger if you pass them a new image that has the same features and the nice thing about this approach is we can identify new images that trigger the neural network instead of just finding a list of existing images so Google is starting to deploy an API for this but again their system relies on the idea that the messages reach Google servers unencrypted and then they can be analyzed by this neural network okay so this is the basic approach now I want to just very briefly identify some of what we care about security considerations alright so first of all we require in these systems that the server has to see the media to process it in these existing systems end to encryption is not possible the scanner has to have some kind of hash corpus or algorithms or maybe a trained neural network that can do this this data is intensely proprietary because I'll show you why but in the hash corpus setting if you get access to those hashes sometimes you can reverse them back into very very low quality pictures so if I give you a hash database full of abusive images the last thing I want to do is reveal that information to you so that's very bad and in the neural network case it's much worse it's possible in some cases to actually would pull images out of a trained neural network and you've seen this before probably if you've ever looked at these kind of neural networks you know you can grab an imagery out of a neural network and the last thing we ever do is we want to do is make one that's trained on abusive images public criminals who have access to the model direct access can easily bypass it they can develop software that takes any image and make sure that it will never trigger the network and similarly providers and third parties now can do abusive things they can instead of just training their model on abusive images they can put political images into their models and make a legal speech and they can now have a system that triggers on that speech as well now just imagine forget about providers being delicious just imagine that a group like 8chan was to get involved in creating imagery that got into this model imagine they wanted to cause problems with these models this is any system that is vulnerable to 8chan griefing the system is not a system you can deploy so these are very bad things I'm just going to very very quickly kind of skip over some of these slides but we remain in a world where we don't know how to do this kind of model evaluation or hash evaluation on encrypted data most of the techniques we have require some kind of multi-party computation or fully homomorphic encryption we can use a variety of different techniques but they're all fairly inefficient and they also have problems with this question of what if the provider has a database that has images that we don't trust if the thing that we're evaluating is a neural network we actually have some results that tell us how efficient it is to evaluate a single image against a neural network and these are small images it turns out that some of the best systems we have take hundreds or thousands of megabytes of communication to do a two-party evaluation of one image against a neural network so the technology is just not ready for that we need more research and on the question which I think is probably the most interesting research question if we're going to do content scanning how do we stop this from turning into an opticon how do we stop governments from using this to essentially turn into a spying system against end-to-end encryption and we just have no idea how to do that how do I know that the system isn't scanning for anything that an abusive government wants to scan for or maybe somebody inside attack or inside the provider wants to scan for the answer is we just have no concept if we want to deploy those systems safely we have to build a system that can actually guarantee that what's being scanned is something you know the database is something that we trust and if we don't do that you know if we can't build that how can we possibly deploy these systems the only possibility is to build basically the largest mass surveillance system that any democratic government has ever built and we just don't want to do that so there's a lot more research that we need to do and it seems like I'm out of time so I'm going to very fortunately reach the end of my slide so thank you very much and I hope some of that was helpful and here to answer any questions you have it was very nice to be here thank you Hey questions and now open you can use the Q&A tab and post the questions and just delete them out and I should be able to keep answering them the first question is from Suman Kar Assume key SQL isn't used in case FBI presumes the case don't they have to share the master key during discovery please So the current approach that I've seen given and proposed is that a provider and here I'll say Apple will store the master key and a message goes through the network and or maybe a group of messages and the FBI comes and says we need all messages sent by this phone number at a given time and at this point Apple is permitted to use its master key and decrypt only those messages and turns over the decrypted messages to government and doesn't have to reveal the master key that is at least the way the United States is trying to offer a guarantee that you know this won't result in mass surveillance that's the proposal that I've seen and so hopefully no we don't have to reveal the master key Okay the next question and I asked he is holding a lightweight server competition will IoT devices be able to implement the standard alcoves faithfully and be descriptive at enough costs to discourage OEMs from implementation? Yeah I mean at this point you know anything that we do anything that we do to I mean of course IoT may not have to do antenna encryption but actually it is right so my my phone today uses a variant of Apple iMessage to do and I think we talked about this in the last session to talk to other devices around the house and to do antenna encryption anything we do to add key escrow is going to significantly increase the weight and the complexity of the protocols that we have to build and this is definitely not something we want to do in areas where lightweight crypto is required so it's another disadvantage of these approaches Third question FBI dropped the child porn case because they didn't want to disclose what is called network investigative techniques which reveal people's identities technology does provide a good counter argument to breaking it Yeah this has been interesting so there is I think when people call about talk about NITs what they're talking about is browser hacks or phone hacks and this has been to some extent the way that investigative agencies have bypassed encryption in the past they give something that is a JavaScript vulnerability in your browser or some kind of drive by vulnerability in your phone their argument this is too expensive and too unreliable for them so they still want surveillance they still want encryption key escrow and so on because they think the costs are too high of using hacking and it's not going to be sustainable however the evidence is pretty good that it's worked well for them so far and of course there have been these kind of proposals people have said hey we will allow hacking if you agree to drop key escrow but that's not how negotiation works certainly they may ask for key escrow but there's no quid pro quo right we're going to end up with probably hacking and key escrow so that's the thing we need to keep in mind Shai Kittress asks how would the Israeli NSO hacking to E2E encryption and he's asking this in the question of context for WhatsApp yeah yeah obviously very good question yeah NSO does exactly this they make zero click now there's zero click attacks they used to be one click attacks now there's zero click attacks on things like Apple iMessage and other kinds of apps and they're really good at this but it's not cheap it's very expensive to get an NSO exploit and target somebody the real downside of end-to-end encryption is that it's only secure if the ends are secure right the idea is that this phone is one end of the communication and your phone is the other end if you get this phone then you have to plain text and so a lot of these NSO type exploits they get access to the phone they are then able to basically dump the entire WhatsApp plain text message database and you're done now what I will say about that is some of the abuses of NSO have been kind of abusive you've seen NSO exploits used against journalists in some countries in ways that clearly do not seem like reasonable law enforcement however what I will say about that is at least the number of cases of this has been fairly minimal whereas if you talk about something like Kiasgro you really have the capability for mass surveillance and NSO style hacking at least limits that kind of mass surveillance somewhat so that's the benefit of living in a world where NSO exists only NSO next question currently India is attempting to send messaging apps through collecting the originator of specific messages that the deem affect the sovereignty of the country in such a situation how do we ensure that individuals of specific groups are not targeted how can we reduce algorithmic bias in such a situation I don't know that there is a way I wish I could tell you that there is a way I think at a certain point right the once I'm a generally politically very distrustful person but my observation over my lifetime is that once you have the capability to do this stuff whether it's intentional or not some kind of bias will creep in and you will see these things abused they may be abused by a small number of participants in the system but they will be abused and the only way to prevent that kind of abuse is to stop the least targeting any sort of targeting in the first place I'm not saying it's a perfect solution but that's all I can offer any other questions what side operations which will be performed for content scanning could be spoofed by having a malicious device which is something else and for something else how will this be solved yeah so some of the proposals right now and I believe snapchat may be deploying this is I push the hash database down to your phone and then your phone has a copy of it and then checks and of course this assumes that I haven't jailbroken my phone and I haven't hacked snapchat of course if I do any of those things disable content scanning and there's nothing the provider can do about it so this is a big weakness and you have to imagine that people who are very serious about using these systems to deploy illegal content will figure out how to deploy a jailbroken snapchat app so this is a big weakness in these systems again I think the best thing to think about here is that these content scanning systems they're not designed to work against people who are very smart they're not designed to work against the kind of people who attend this conference they are designed to scan and to catch people who are non-technical and who are dumb enough to send their content through an app that is well known to have scanning and not to do anything to evade it I think that's how governments are thinking about this they're thinking that's good enough but you're absolutely right from a technical perspective it's a very hard problem to solve Sumana asks Facebook paid a cybersecurity firm six figures to develop a zero-day entails in order to unmask a man who spent years extorting children do you think this is a fair compromise to transfer offenders without breaking encryption I mean I would if the alternative is mass surveillance I would much rather that's how things are done I mean it's you know this is a place where we're going to have to find a balance we have to find a balance between catching the kind of people who do things like that and building a system where there is no protection for anybody I just I think that once we make the compromise of there is a master backdoor that compromise we can never go back from it it is something that is going to expand and the usage is going to leave our control the only control we have is do we have a the ability to say no to that backdoor and then revert to targeted approaches like what you just described I'm a little dogmatic about that but that's how I feel any more questions okay do you foresee a possibility at all that CCM and such beneficial content scanning can be affected through processing online devices I think it can be there are some downsides to it here's the biggest downside just from a technical perspective in the case of photo DNA which is the hashing algorithm my students and I have been looking at this algorithm to see whether it can be abused and we can't we can't do it because photo DNA is secret Microsoft owns it and will only release the algorithm under NDA now you have to ask yourself why is Microsoft keeping a hashing algorithm secret the answer is because they don't trust it they think that if they reveal the technical details of that hashing algorithm to the world it will be broken and people will figure out how to bypass it and or trigger it on non-malicious content and all of the bad things that can happen that to me indicates that Microsoft and others are not very confident in the robustness of this technology they're relying on security through obscurity to keep it working now imagine a world where we put photo DNA inside of every app and the code is there and anybody can reverse engineer it and take it out does photo DNA still work in that world I don't think Microsoft knows I don't think anyone knows and that's a concern for me Any more questions we have some more time for Mr. Green I'll wait for 5 seconds and ask any panelists panelists were already there you're not free to ask questions directly Renal Any more questions Alright I will ask a few then Is it possible for satisfying some of the intelligence agencies request by using metadata for messaging apps I think so yeah I think that metadata reveals a lot and I think that currently is being used to a great effect we have the NSA as a line and one of the Snowden's lines that said we kill people based on metadata which is not a great thing I'm not endorsing that, that's awful at the same time they're able to come to enough confidence that they can perform military options based on metadata there's a lot about what you do the problem here is that this is a world where having more information is always better if metadata works well but having the content of data would work even better then you're always going to feel pressure for people to get the content too it's inexorable there's no compromise that anyone will be happy with in this world unfortunately You had also mentioned about the Carnegie's encryption working group paper they basically said one of the parts that one could take is go for the cell phones and not touch data at motion is that a viable alternative at least in terms of dissolving the standoff to some extent I wish that we had gotten there I thought we were resolving in that direction I thought we were resolving the direction of hardware only and data and motion is a problem for some other day however the content scanning debate really undid some of that is inherently data and motion because we're talking about messaging with content scanning but it's such a powerful requirement that we need access to every message just to scan it but every message that it really turns the debate upside down it seemed like we were getting to a place and now everything is thrown apart so we don't know what to do Is there anything that you want to add upon Ray Ozzie's proposal of burning the hardware that just blows up Yeah, I don't love it you know my big objection by the way Ray Ozzie has this proposal where he says we can have phones that there's a master code that you can enter into a phone which will unlock it but there's a protection which is there's a hardware chip that will burn a fuse or something when that happens so you'll at least always know there's no way to scan somebody's phone without them having a dead phone and Ray Ozzie bases this on the idea that we can build secure hardware into phones I would comment that we've spent a lot of time trying to build secure hardware in phones and we continue to fail and up until very recently there was a major vulnerability in Apple's secure enclave processor which is the main hardware processor inside of phones that was allowing people to brute-force pass codes which they should not have been able to do so I love Ray Ozzie's idea I'm skeptical that that mechanism will work as well as Ozzie thinks it will I just want to make a plug for some research we did if you go to securephones.io my students and I spent last summer writing about 110 page documents on everything we know about security mechanisms in existing Apple and Android phones, it's not complete but if you want to learn a lot more about how well this hardware works and what Apple claims it can do that's a great place to go securephones.io So Shubham Jao asks while the keys used for encryption of the messages are different can using a Cypher block chaining mode like ECB for encrypting and when messages help directing patterns of messages and correlating Cypher texts? Yeah I think that the encryption modes right now are all pretty good at doing what they have to do and I see so I think you're saying can we make the encryption weaker is just trying to understand can we leak some data or just trying Yeah Apple does have these deterministic they call them convergent encryption modes where if you encrypt the same message twice then you get the same Cypher text that could leak some information but that sounds awful to me so every time I say okay we're going to leak some information that is recognizable that seems I don't like systems that use technical weaknesses to make to solve policy problems but at the end of the day we have to tackle this head-on and say look what is government want how reasonable is it to give it to them and then let's build the system that does exactly what they want let's not let the cryptography dictate what policies we are going to build let's build the policies that we need to build and so generally speaking I think leaking data deliberately is not necessarily going to satisfy anyone and it may be actually very harmful I had a question on your paper 110 pitch paper so one of that is is it possible is that how celebrate and vendors use it for hacking phones in general when they talk about doing before boot, after boot kind of a model Yeah Yeah There is this notion there's this notion of before first unlock and after first unlock and I think that's what you're asking about which is when you unlock your phone a bunch of encryption keys stay in RAM and they don't necessarily go away everything that we did last summer was to try to answer the question of how celebrate is getting access to data do they have some way of breaking encryption breaking the secure enclave processor and Apple phones and the answer is as far as we can tell once you have an unlocked phone that's in the after first unlock state the keys are in RAM if you can get past the pin code using software techniques software protections then yeah you can get access to data and there's very little encryption on the phone so that's our theory is right now that's how great shift and celebrate getting access to phones There are a couple of questions on YouTube just let me just go mute it and then go find it David I couldn't find the YouTube link or questions on it will you be able to find it and paste it so the questions are are backdoors means of accessing potentials for example introducing vulnerability to software updates is this different from the SQL yeah the introducing vulnerability to software updates is a huge problem one of the problems we have right now is that all of this end-end encryption assumes that we trust the client software you trust the endpoints you trust the client software if you don't trust the client software things are very bad and right now we live in a model where Apple and Google and WhatsApp we trust them to push honest software but we assume governments can show up and demand plain text but they can't force them to push software updates this is not a world we really can live in but the techniques we have to prevent that are things like open source and reproducible builds but even there we don't have perfect solutions and I'm very very nervous that we'll move into a world where that is a common technique for governments to attack software update systems and push malicious updates and we won't have good defenses the next question is by Ananshi Kipputu he asks the content filtering for CSAM can be easily bypassed by adding another layer of encryption can I just you're not missing anything it's like I said if you are determined to get your data through these systems if you're technically savvy and you're even mildly technically savvy they do nothing and I would also add that many of these content scanning systems here in the US Facebook 97% of the hits that Facebook's hashing algorithm records 97% of them are outside of the United States and a vast of them are basically beyond any sort of law enforcement response so most of what these hashing algorithms do is they allow Facebook to detect and shut down accounts they don't even have a major impact on the actual production of this kind of media so yeah I think one way you can look at this is you could say that this is designed to catch the low hanging fruit of people who are technically incompetent and are not very advanced the other thing you could say about these algorithms is that they don't work they're not stopping people from sharing this material the material is being shared more than ever and people don't care about being caught they don't care enough to encrypt their files before they send them through a Facebook messenger simply because there are so few consequences to them these approaches just don't work very well so you know it's sort of like if you have criminals running around in broad daylight then you know the problem is that maybe your laws and your police aren't working I think that's one way to look at this right now yeah you're right you could bypass them and people are choosing not to okay we have a few more minutes any other questions from Mr. Green you can use the Q&A tab or put it on YouTube what type of policy is essential to combat the large scale surveillance the government are attempting to implement as you said you need the policy to define the software it's very hard yeah no this is a hard problem and I wish somebody would solve it because this is what we need to do before we deploy things you know one of the things we need is we need to have some sort of notion of what is it that we want so for example one policy might be that we can absolutely decrypt some data under warrant but we need to be able to publish the number of such warrants I mean that at least would provide some immediate visibility into whether there's surveillance or not it's very hard to do that enforce that technically other policies we might include are that you know the warrants have to be digitally produced and maybe someday they become public that's not something that police are very comfortable with right now but that might be something we could do again and if we're moving over to the CSAM detection world then the question is let's say we have a database just checking that those images are actually CSAM images it's not something you can just release to the whole world to verify do you have an agency that is well widely trusted who checks that what's going into that hash database or that model is actually what you expect it to be there need to be systems in place to do this and we just don't have any idea what those systems look like two more minutes any other questions David are there any other questions that came on youtube can you please paste them here no that's all on youtube one more minute these have been really good questions by the way so don't feel bad if you're not giving anymore this is great okay I have two last questions and I will let you go after that so here is my difficult question for you not really encryption but about policy what if so the first question is about signal do you really think it's a good idea to bring in payments but equally cryptocurrencies into its product line given that the comments are much more hostile to cryptocurrencies as it is they are much hostile to encryption e2e and bringing crypto means all this stuff about money laundering and all that kind of stuff do you really think it's a good idea yeah no I think it's a very bad idea I want to just comment here full disclosure I you know co-invented Zcash and you know have a little bit of conflict here because you know when you talk about payments everyone will say hey you know he's talking about his own currency that he helped invent and that's true I do want to say this though which is that signal is a very clean encrypted messaging story and it is a target of governments and they want to shut it down they would use any tool in their arsenal to shut it down and when signal is using a cryptocurrency that is some pre-mined cryptocurrency that's also private they are moving out of a world where we can say look it's just messaging and it's protected by the US First Amendment and this and that to a world where suddenly they're doing very different things and I think it's not a good place for them to be it's not a safe place for signal to be and I really wish they weren't doing it if they went ahead and released a currency and just put it in another app that'd be great it's putting it in signal that makes me nervous last question um what if governments actually put a gun like in India what has happened is that they're basically saying if you're not complying to the rules and regulations that we're asking you we're going to put your executors on jail do you really think it worked in any other place before you know any place can reach that point and it's not so it's not so far you know the argument I've made in the US is if you pass legislation that lets you do that here in the US the people will move overseas and you'll start seeing encrypted messaging apps out of Switzerland or India or wherever but it's possible we'll end up in a world where every government does this and I'm not sure what the answer is maybe it's some kind of decentralized system it certainly won't be good for anyone okay I think we have answered everything um thank you Mr. Green for your time thank you nice talking to you too thanks very much alright right the next session can you make a panel to have a bunch of announcements to make alright okay so we have an interesting session which will be held slightly differently I will just read through what the session is going to be Uzma here is going to present a session on how the implemented live location on WhatsApp and also some interesting technical details about the Diffie-Helman and the Ratchett algorithm just to re-emphasize this is a session that is held under is actually a butter figure but