 Thanks for joining us today. My name is Axel Armbach. I'm a fellow here this year at the Berkman Center and at CITP in Princeton. This is a somewhat unusual setting. After the talk, the day before yesterday, we found out that some of the backup or some of the stream had been gone. So I'll do the first ten minutes of the talk again. I was asked to do that and then afterwards we will gradually go into the talk, into the real talk. So you'll see a different shirt and you'll see audience and that will be fine. So today I'll be talking about government hacking, both in the law enforcement and in the intelligence space. And the recent response or the response a couple of years ago by the German constitutional court to call for a constitutional right to IT security. I start the presentation with this quote from 1971. The quote reads, in almost every issue of the weekly computer world, which is magazine, is an article detailing a case of computer fraud, embezzlement or sabotage. Over 100 different articles had been written on this issue from the mid-1971 until 1972, the sources down there on the slide. And this goes only to tell that even though with the NSA revelations may possible by Minister Snowden and with some law enforcement hacking issues, cases that have been popped up in recent years, we think that this is a very new issue but it has already been around for 40 years. But legislators are only now thinking about this issue and how to respond. So we have to realize that the technical reality is 40 years ahead of the legislative debate and that is something that I want to focus on today. So hacking, government hacking has been around for a long time but it has really accelerated in the recent years, especially the last 10 to 15 years and especially the last couple of years. And this has all to do with social technical changes that everybody has heard a lot about but we move our communications, our information into the web and recently into the cloud. And there was this interesting moment in the mid-90s that is now commonly known as the crypto wars. And the crypto wars were all about the FBI and the NSA and other government authorities all around the world were saying that communications were going dark, moving behind, for example, HTTPS encryption and that made it much harder for government authorities to do the conventional types of surveillance through wiretaps. Now this led to these crypto wars where one part of civil society, for example, but also industry, financial industries pleaded for strong cryptography to protect communications but on the other hand you had government authorities in law enforcement and intelligence saying no, we should not allow this strong encryption to happen because that disables our surveillance capabilities. Now the interesting moment was the crypto wars and the conventional wisdom is that the crypto wars had been won by industry and civil society. So there is this famous book by Stephen Levy called Crypto that details or that outlines the whole crypto wars and that under title is quite beautiful. It says how the code rebels beat the government saving privacy in the digital age. Now for over a decade this has been the conventional wisdom. It has been very much part of, you know, technical and legal debates that the crypto wars were won and encryption saves the day. Well, of course with the recent revelations made possible by Mr. Snowden, for example the NSA bull run revelations but also last week, I'll talk a bit more about that. We see that this is not really the case and we're actually a bit more in a situation like this beautiful picture that I found where you have a clunky internet infrastructure and the code rebels going up and enjoying the excitement of the ride and then finding at the end that there's an obstacle between their legs when they're down and this picture we remember freedom plays with that nostalgia. I think it's a beautiful artwork. That's more realistic in the situation that we are today. Only last week Glenn Greenwald, the journalist, his outfit with Ryan Gallagher, the new outfit of the intercept published new slides on how the NSA plans to infect millions of computer with malware. These are all about network attacks. At the end of December, Der Spiegel, Laura Poitras, Jake Appelbaum and others published the tailored access operations catalog, which is much more about hacking into devices. And now we know that the capabilities of these intelligence agencies all around the world are enormous and that all vulnerabilities and communications are basically being exploited. Also in the law enforcement space, recent quote from the Washington Post about the FBI and it reads the most powerful FBI surveillance software can cover the download files, photographs, stored emails, gather real time images by activating cameras connected to computers, say court documents and people with familiar with this technology. So this is happening and the plausible deniability about government hacking that has been very much present in policy debates, it's now gone. We know that this is happening and that it's increasing and that it is very problematic. But today, we'll talk about Western governments. So how should we understand the hacking efforts of Western governments and how should we respond to them? And the goal for today's talk at the end will be to co-create and think about and have you all think about an agenda for research policy and also activism. And activism I think is very important on the short term to raise the importance of the issue and to do fact finding. So today we'll not be directly about something that has been very much in the news lately and the citizen lab of Berkman friend Ron Debert has done some fantastic work in this space, which is all about governments, non-Western governments and I'm fully aware of the contentious distinction between that. But here there's been a lot of coverage about how these non-Western governments have been purchasing hacking software made by Western companies. This is a slide on a hacking team, one of those companies, and the suspected governments that use their RCS software all around the world. Well you see some pretty interesting countries there, Saudi Arabia, Kazakhstan, Turkey, Nigeria, even Ethiopia. The Ethiopian case has been covered by the Washington Post lately. But today it's not directly about that. Why? Because Western governments have a long tradition of dual use regulation. So arms treaties not selling your weapons or your digital weapons for that matter to non-Western governments and authoritarian regimes. There's already a lot of great work being done, second reason. And the third reason and I think that's really at the heart of this issue is that it's super, I mean it's very attractive for Western governments and Western politicians to point their finger at these companies and authoritarian regimes. It's easy to condemn for Hagamons basically other governments, but what I want to talk about today is that Western governments themselves are very much engaged in this. And we have to have a much more rigorous debate on this. Okay, so today the outline is I will first discuss some hacking cases by Western governments. Then I will highlight, you know, it could be 11, it could be 37, but I want to specifically focus on 11 problems with government hacking. I have a longer blog post on that if you want to read more about that which is also on the Berkman website where you found this talk if you want to do more reading. Then I'll talk about this new constitutional right to IT security which was proposed by, which has been installed by the German Federal Constitutional Court. And you already see traces in the European Court of Human Rights of that being accepted. And finally I want to discuss with the audience, and I have discussed with the audience these elements of a research agenda and a discussion. All right, so first these hacking cases by Western governments. There was not so long ago, about a month ago, this fantastic conference put together by Chris Sugoyen in Yale. The stream is online. If you're interested, you should definitely take a look. I had a bunch of great speakers and I was there as well. And some, you know, considerable parts of the talk are inspired by the speakers and I tried to attribute where I could. But definitely worth a look if you want to have more information is this law enforcement and hacking panel conference at Yale. There I discussed two of these cases, but here I have a lot more time to go into detail. So let's just separate three cases of government hacking, law enforcement investigation, botnet prosecution and mitigation, a Dutch case, and this ubiquitous intelligence gathering. Okay, so let's start with law enforcement investigations. In the U.S., this has been going on for quite some time, but we know very little about this. And only a couple of months ago we got insight into a DFBI requesting a warrant from a magistrate judge to hack into the computer of a certain person or because they suspected that from this particular device, they suspected from this particular device somebody using somebody else's credentials to log into internet banking environment and do some transactions there. They wanted to hack into the device to find out who this person was and whether they could find the suspect. And that was exactly the problem with this case. And magistrate Judge Smith, who also spoke at this Yale conference in April 2013, he turned this FBI hacking request down. And the three central reasons why he did this is that the FBI wanted to hack into a device, but they didn't really know the identity of the suspect. That's why they wanted to hack into the computer, and they didn't even know the location of the device. So they wanted to do a search for a device and off a device. They wanted to search the device they thought had accessed this account, but they weren't sure. And whenever they got this warrant, they also wanted to search the computer itself. And they're usually problematic aspects of this warrant manifest themselves because this basically comes very close, Judge Smith said, to a general warrant. You don't know the identity of the suspect, so you run the risk of searching for the computer of somebody that is very innocent and has not even been suspected of a crime, and you don't even know the location. So this computer could be anywhere in the world, and this computer could be at the public space, maybe a public library, for example, a computer that many people use or in an internet cafe rather than one person. And he said, Judge Smith ruled that that does not satisfy the particularity requirement of the Fourth Amendment. And the third reason was also that what the FBI wanted to do is look for IP addresses but also conduct video surveillance. They wanted to turn on the webcam on this particular computer and look whoever was using it. And in the Texas District there is strong Fourth Amendment requirements for video surveillance that hadn't been met. So this is a rare case of a warrant coming out. At the same Yale Conference, Professor Donahue of Georgetown said, well, I've been researching, I've been trying to research this issue, but there's a very big problem because I found dozens of cases from California to the New York District where the FBI requested warrants, but they were sealed, and Judge Smith reiterated this point. They both pointed that there is no public scrutiny possible of this practice by the FBI when all those court cases are sealed. So we don't have an idea of the scope and how often this is being done because these court cases are sealed. This is a big problem that needs to be addressed. Okay, second case. The Bundes-Trojaner. This is a case that already started public debate in Germany about eight years ago. So one of the states of the federal government, the Federal Republic of Germany, Nordrhein-Westfalen, one of the states wanted to legislate the use of government malware by law enforcement agencies and made a law. And this law was challenged ultimately at the Supreme Court in Germany, the Federal Constitutional Court. And here is the Federal Constitutional Court. You see their funky hats. And what was so nice about this case is that the Bundesverfassungsgericht invited three computer security professors and one computer security expert from the Chaos Computer Club to be part of the court proceedings, testing the validity of this law against the German Constitution. And this Bundesverfassungsgericht, the German Supreme Court, really went to great lengths to understand the technical details. And ultimately, and I'll talk a bit more about the ruling later, but ultimately in 2008 the Bundesverfassungsgericht ruled that the German state law was inviolate, so was breaching or violating the German Constitution and came up with this new constitutional right to IT security. Very interesting case. What I want to talk about now is that in 2011 the Chaos Computer Club got hold of a piece of government malware. So even though in 2008 the German Constitutional Court ruled that government hacking violates the Constitution under the specific law that had been installed, it found three years later the Chaos Computer Club found government malware in the wild. They were able to reverse engineer it and this secretly deployed government malware turned out to be very problematic. And this process of reverse engineering is highly instructive for us to better understand government hacking in the law enforcement space. So let me tell you some technical details about the malware itself. So the first very important aspect of this is that the government made false claims about the malware itself. The government always held that even in the Constitutional Court case that their government malware, their trojum was used only for source wiretapping. So only for listening into the communications between the hacked device and other devices, basically a wiretap. But what the German, the Chaos Computer Club found was that in the malware the capability was already built into it to do remote searches. And a remote search is something different than a wiretap because that's when you actually go into a device and search on its hard drives, etc., for more information. Not a wiretap when something is communicated but actually looking a bit similar to a house search, what's on the computer? So the government held, no this is only for wiretapping, but the Chaos Computer Club found, no, they are also able to use this for remote searching. Another false claim by the government was that we will always, our government malware will always be case specific. So our malware is custom made and we have quality audits every time we deploy one specific piece of malware. Well, this is also something that the Chaos Computer Club was able to debunk because they found several pieces of malware which were exactly the same, not custom made at all. And then most critically the malware had a lot of deep security flaws. So what happened? Most importantly I think it was fairly badly encrypted and some things weren't even encrypted. So try to imagine that the government installs a virus or a piece of malware on your computer and then is able to, you know, from the law enforcement offices give the piece of malware on your computer commands. Now these commands that travel all over the internet until they reach your computer, the piece of malware, were unencrypted. So if you were able to intercept those commands, you were actually able to change the malware and to change the commands given to the malware as a man in the middle attacker. So try to let that rest in for a moment and think about its implications. The implications are that, for example, you could better understand what kind of commands law enforcement gives to the computer but you could also modify those commands. So you could actually tamper with the malware in a way that lets you as a third person, so as a man in the middle attacker, for example, do other stuff on the computer of an end user. So this is highly problematic. A second thing you could do is impersonate those authorities. So you couldn't, you could even pretend as a third person that you are the government and use the malware to give all sorts of commands to that computer or maybe even to use the malware yourself in other cases. So hack other systems and do that. And then a third very big problem is that as a man in the middle attacker, you could actually go back, use the malware to hack law enforcement systems. So you could actually, and this is, I mean, the consequences of this are enormous, of course. You could actually go, use the malware and go into the systems of law enforcement itself and then change all sorts of aspects of government law enforcement databases. For example, I mean, the implications you could think of are enormous. So all these problems emerged. Highly insecure government malware deployed even though the government had a constitutional court ruling against it. You know, government malware not being specific, but generally used with all sorts of capabilities already built in. Highly problematic case, highly secretively deployed and only because the Chaos Computer Club reverse engineered it, we've been able to see its implications. I'll talk more about it later, but highly problematic. Okay, botnets, different case. And for botnets, we look to the Netherlands, but this happens all over the world. But then the Dutch case is really interesting because it shows, you know, the information that has emerged is just very rich. So if you would have seen on your computer this warning screen, somewhere in 2000, October 25th, 2010, it meant that your computer was infected with the breeder law botnet and that the government had infected your computer with a small piece of code to spur whenever you turned your computer on this warning screen. And this warning screen is, here's the police logo, public prosecutor, the Dutch cert and a company that actually made this malware. So this is a case in which a lot of computers all around the world were infected with a botnet and where the Dutch government used the botnet to mitigate the botnet. So a botnet is very basically a server, a command and control server somewhere in the world that has infected lots of, as you call it, bot clients. And this creates this zombie network in which this command and control server and the bot herder that controls it can send out commands for, you know, search for credit cards, et cetera, all over the world. Botnets are a really serious problem for internet security and for crime. It's not at all to understate the problem here. The Dutch government had said that 30 million devices all over the world had been infected. They exaggerated this number at least by an order of magnitude to, you know, hype up the seriousness of this case. But definitely this was a large botnet. And so, yeah, people were, there was a lot of attention on this also from other countries to do something about this. And the Dutch law enforcement agencies took a pretty bold approach. So some of the technical aspects here is that, as I already said, the Dutch law enforcement agency claimed that 30 million devices were 30 million devices. So 30 were infected, but it was rather more like one and a half million, still a lot. And to prosecute and to mitigate the botnet, the Dutch law enforcement agencies did two hacks. One, they hacked into this command and control server. They did this to find out who was administrating the botnet and to prosecute him. And the second hack was a hack into all the computers around the world that were infected with the botnet in order to install a closed source, unexpected piece of code on one and a half million computers all around the world that would spur this warning screen. But whenever this happened for years, nobody had seen the code. Nobody actually knew what capabilities apart from spurring this warning screen were part of the code. And users, on the other hand, would see this, would go to this link, this link mentioned here, and their systems would automatically be updated. That, of course, had a lot of security implications. Bits of freedom at Dutch NGO, where I, not at the time anymore, but where I used to be at, did a lot of freedom of information requests for the code. And finally, well, the freedom of information request wasn't honored, but the government released the code itself. And, well, if you see it, it's actually a pretty, you know, it's not that interesting a code. It's just basically asking for the windowshell to open and then for computers to serve to this URL, this URL would then spur this warning screen. Nothing more. It's language Delphi-6, I think, not particularly interesting. But what is interesting is, for one, that the warning screen wasn't encrypted over SSL. So, basically, this opened up to phishing attacks where you could actually impersonate a website or do whatever. You could listen in to all the computers that were visiting this warning screen and actually assume that they had been infected by this botnet. And the code was not even signed. So, basically, when you went to this webpage and clicked on the website or went to the website and you saw the link, and that would then, you know, execute this code, the code wasn't even signed. So, there was no MD5 checks on. I mean, cut a long story short, everybody could have inserted a different link in there or maybe antivirus companies from a bit more rogue nature could actually do all sorts of bad things there. So, again, a case of law enforcement hacking in which security was a pretty serious issue. Even after the botnet was mitigated, within weeks, the botnet was already functioning again, which also raises very important questions about botnet mitigations itself. And the botnet moved from a centralized command and control server to a peer-to-peer infrastructure, making it even much more hard to mitigate the botnet. Third case, ubiquitous intelligence gathering. Well, this has been over the news a lot in the last couple of weeks. There was the end of December where their spiegel published the tailored access operations catalog, which showed us that the capabilities of especially the NSA and the GCHQ in Britain are pretty far advanced. And then there was a week ago a new slide released on the, this is then the quantum catalog. And, well, there's a lot of information here. I won't talk about everything, but particularly interesting is quantum DNS. You know, DNS injection is done, all sorts of men in the side techniques, men in the middle techniques. And what is very interesting is that a lot of these operations have already been operational, or a lot of these programs have already been operational for, you know, for about 10 years. Here's Quantum Sky for the denial of access to a web page, already operating since 2004, et cetera. So already for, and basically this is dirty hacking techniques. It's not particularly special, but it's used and already for a very long time. And, you know, with all the revelations, I guess the basic point here is that, well, Internet security is in a pretty deplorable state. And also that the fact that it is in a deplorable state is being exploited massively by intelligence agencies. And once again, you always need to tell this, I mean, this isn't only the United States. When these vulnerabilities exist, it's quite probable that they're also used by, you know, a range of other countries all over the world, and not to mention cyber criminals. So a lot of very, very deep problems and lack of all sorts of means of oversight and legal protection. So that's what I want to talk about a little bit more. You know, there's a whole bunch, as this is such a new issue, and as legislators have only in recent years started to, you know, in Europe, started to think about this, and only in recent months in the United States, the technical capabilities are 40 years ahead of the mindset of legislators. So that's something to keep in mind. And basically, just very briefly, I'll talk about, you know, 11, but it could be, you know, the list just goes on and on. One very basic problem for cyber crime mitigation is that we have no reliable data. We have no idea how much the costs are of cyber crime, save the, you know, the annual report of Simantec that of course has a huge interest in pumping up these damages. They often, for example, they include copyright infringements from music downloaded into their reports, which is numbers they directly get from the entertainment industry, that kind of stuff. It's really dodgy, it's not reliable, and we don't really have reliable data to make evidence-based policy. Very deep problem. Then, as we already saw in the German case, there's no clear separation, there's no clear technical separation between wiretap and search. So most of the malware that is employed already has all sorts of capabilities built into them, and you can argue whether doing a wiretap on somebody's system is actually a wiretap. No, a wiretap would be looking at the wire and not at the system. But now, a lot of this government malware is targeted at the system to perform a wiretap but built in with all these new capabilities. So this old legal conceptual separation between wiretap and search is something we really need to think about and is very problematic. And with all these systemic lying by law enforcement agencies and no laws, we have really hard time to do proper judicial, so in the courts, and policy oversight. And we already saw this in Majesty Judge Smith's and Professor Donews' comments at the conference, but oversight is really hard and where we have no reliable data on the actual scale of the problem, we have no reliable data of the actual practices either. Well, insecure malware, both in the Dutch, but certainly in the reverse engineer German, Bundes-Trojaner case, is an information security disaster. You know, in Germany we saw that the malware employed could actually not only be subverted, but it could also be used to attack law enforcement systems. So this is like, you know, acid of the most serious sorts. This is something that you really don't want to touch if you don't know what you're doing. And then very importantly, I think, allowing government hacking to occur creates really bad security incentives. So we all have a benefit with information security in our network communications, but creating this possibility for governments to hack creates these very bad incentives. Just as a short aside, there's about three ways to enter a device. One is through social engineering. So for example, phishing attacks. Another one is through installing a box at an ISP that enables a man in the middle attack at the ISP level. And the third is to do it at your operating system. Something we have already seen in the NSAID disclosures that Windows Update is actually used to report the vulnerabilities in a Microsoft Windows setting to the NSA for them to determine their attack techniques. So social engineering of an individual is one thing, but placing a box like Finspy, it's a box offered by this company called Gamma International from the UK, or involving operating system vendors in this process makes the whole ecosystem really vulnerable to attack. So this is a big problem. And the question we should ask ourselves is, will Microsoft, Apple and Google, will they be forced by the government to comply with the request to actually report back security vulnerabilities or even insert backdoors in their system? On the other hand, of course, there's the ISP. If the ISP installs a box in its network which enables, like all their millions of subscribers to be hacked into, of course, if somebody hacks that particular box, then you have a really serious problem. Another issue is of course with antivirus. What is antivirus going to do when they flag government malware? Will they be compelled to let it through? Firewall intrusion detection companies, they all have a really, really hard problem to face in the coming years, or maybe they're already faced with this for 10 years, but we don't know about it. Sixth problem, the scope is completely undebated. So are we hacking into a user, into a device, into a router, into the ISP? Are we hacking botnets? Are we just hacking the entire world? Nobody really has talked about this issue, and nobody really has thought about how to tailor these capabilities to specific context. And this is of course a very difficult question, and I'll talk a bit more about it later. Seventh problem is a very obvious one. It's the one of jurisdiction. So are these capabilities only to be used within your borders or across borders, especially in a law enforcement space? Russia and China have not yet signed up to the cybercrime convention, because of territorial sovereignty issues, and you can already imagine what the Dutch agencies that hacked into a computer in Armenia to prosecute this particular bot herder, this Armenian guy, you can already imagine what people in Armenia will be saying next time some attack or some botnet originates in the Netherlands. And Dutch authorities haven't really thought about this until bits of freedom actually raised these issues, and they were thinking, oh no, wait a minute, we're not really happy with Armenia hacking into our systems, giving them. So it's a geopolitical Pandora's box, right? And this can already be witnessed in this process of the cybercrime convention. The cybercrime convention has its issues, definitely, but it's the one, the single most authoritative legal document for actually harmonizing and working together to combat this problem of cybercrime. But for geopolitical tensions, and if we don't solve them, this will be a convention that's only locally adopted, which creates all these safe havens for crime. So the fact that we haven't discussed this yet is a major problem because it creates a lot of distrust. Okay, and then there's, of course, the constitutional issue. In the wake of the Snowden Disclosures, there has been a lot of debate, and I've already been working on this for two years, about the constitutional scope. So it's a well-known fact maybe here in the United States, but when we presented our work in European Parliament like two years ago about the fact that the Fourth Amendment doesn't actually protect non-U.S. citizens, people really were scared. They were thinking, hey, wait a minute, these are our friends and our allies, but we had to tell them that the legal reality is, I mean, there's all sorts of political reasons to be friends, but the legal reality is that when I move, I'm a Dutch and Danish national, and when I live in Europe, I have no protection whatsoever under the U.S. Constitution, whereas a lot of European human rights treaties are universal in scope. So this protection across borders is interesting. And I mean, I live here not so far from here, but when I VPN my internet traffic to Amsterdam, do I get the same constitutional protection against the government rating my house here in Cambridge than them hacking into my system because they think it's in Amsterdam? Those are all super interesting and very tough questions that we need to think about. Now, another issue that was talked about a lot at the GL conference is parallel construction. This has also been disclosed in the wake of the Snowman revelations, but that's where the NSA, with its capabilities, hacks into a system, passes on a message to the FBI and says, you know, you might want to have a look at this particular system. We can't really tell you why. And then the FBI constructs or rebrands this lead and creates the evidence of its own. This is a very old issue already in law enforcement. It happens all over the world, we guess, or we don't know. But in the hacking context, it's really exacerbated and augmented. There's a number of very serious legal commentators actually claiming that in the Dredpire Roberts case, this guy that purportedly administered it, the Silk Road, an unending service on the Tor network, was actually caught in this way. The NSA may have seen something on its networks and then passed on a tip to the FBI that hacked into its systems. This is, by the way, only speculated on legal blogs. This still has to emerge in reality, but if I were Dredpire Roberts' defense lawyer, I would definitely know where to look. Then the fundamental question is, of course, is this hacking really necessary? I'll talk in a short moment about Utopia, which was like this new Silk Road, a new unhidden service, when the Silk Road was shut down and the Dutch police actually investigated and prosecuted the administrators, not by hacking but by good old undercover techniques and engaging with them on the service itself. Do you really need hacking given all these fundamental problems? So there's a lot more, and that's why at this conference I called Government Hacking, it's a bit like a hydrate. It's a Greek mythological monster. You cut off one hat and you think you have an apparent benefit, but a number of problematic policy issues already emerge. With that, let's take a closer look at the German case about the constitutional right for IT security. We're right now living in this great experiment where we put our most sensitive information and data on our systems. We've seen that it's really vulnerable to security attacks and the German Federal Constitutional Court stepped in to respond to this situation, this great experiment that we live in. This was in the Bundes-Trojaner case, where the Federal Constitutional Court took a closer look at what actually is qualitatively, what is actually happening when you're talking about rights. So they ruled in this case, and here's the source, that IT systems are particularly sensitive and they separated systems from communication and data, and they said that systems, they deserve particular protection because we structure our lives on our systems. I mean everybody here has a computer on his lap and you not only have your work documents, but you have your holiday pictures, your email, everything is either stored on this box. We even structure it. We make it ready for law enforcement to actually look into our lives. This is literal words of the court. They say this is an all-stop shop for government access and you risk making it way too easy to do surveillance. Even they considered network communication, so the cloud. They said the cloud, even though you don't store that information in your particular service, the cloud really exacerbates the privacy intrusion. Why is this the case? Because when you hack into a system, very often third parties will connect to it. You will learn a lot about these third parties and you centralize a lot of your data, even more than on your own machine in the cloud. So this was a particularly tax-savvy court ruling, one could say, and four information security professors were witness to the court proceedings. One member of the KS Computer Club, like an anarchistic hacker, was part of these constitutional court rulings, which was really an example of how you should do this kind of stuff. And then there's also the fact that with all these technical details in mind, they said, well, hacking IT systems that violates the core of privacy, the core of personality. So that's really where our most sensitive stuff is happening these days. So hacking into IT systems goes beyond a wiretap of your communication and it even goes beyond the protection of your home. So here's your home. And they said for all kinds of reasons, our IT systems are even more intrusive than our homes. Very interesting. So that's why they accepted this human right to the confidentiality and integrity of IT systems. And the computer science people here in the room will recognize the CIA triad here. So both confidentiality and integrity is protected. This confidentiality protection has a really broad scope. It is for all general storage devices. So it's not only like your PC or your laptop. They thought about the Internet of Things. And they thought about, in this court case, also about RAM memory. And they thought, well, and about the cloud. And they said that whenever there's a general storage purpose, it deserves this kind of protection. So it could well be that in Germany in a year or two, your toaster has constitutional protection. It's quite interesting. And this is regardless of the technical expertise, the information or operational security practices of the user. So that's also really interesting. And your integrity is also protected. So manipulation of data is also covered. OK, this right is not absolute, just as privacy is not absolute, but a limitation or a breach of this right must adhere to the strictest legal criteria and stricter than a house search. I think that's really interesting. The exceptions are, then again, the foundations of the state. So it seems as if national security is explicitly allowed and the prevention of damages to the foundations of the state. But these preventative acts need to have a high probability of occurrence. So you can't just claim terrorism and everything's happened, but you need to particularly realize it. And then very interesting, this core of the private life, which is a concept in German jurisprudence that is really the most intimate space of your privacy. I won't go too much into it. But if you find really intimate data, you have to delete it immediately. So that's also very interesting. Unfortunately, the court didn't really expand on this. So we don't really know how this will flash out in detail. But at least it gave the basis for future court cases to look at this and see, well, this was really intimate data that it should have been deleted. It cannot be part of court proceedings. All right. This is Germany. But how about the European Court of Human Rights? Well, I'll talk in a bit about a case called I versus Finland where we see the first traces of this emerging. But most critically, the European Court of Human Rights was a couple of weeks ago fast-tracked a case in which both, in which the Snowden revelations are central. So we will see surprisingly quickly, like in a year or maybe a bit longer, but not much longer, we will see an actual court ruling of the European Court of Human Rights about the Snowden revelations. And that's really interesting. I included this picture because many people don't realize, especially here in the United States, that the European Court of Human Rights is of the Council of Europe and not of the European Union. So this includes Turkey, Ukraine, and of course, most interestingly, Russia as well. So when there's a ruling about this, whenever this happens, maybe next year or in a couple of years, Russia will also, yes. About Snowden? Yeah. So an English set of NGOs and Konstanz Kurz, Chaos Communication Club, a German computer science researcher, filed a court case in England against the GCHQ and against the Temporar program and against the upstream program through which the GCHQ gets data from the NSA. And it's just as an aside, nearly all the interesting surveillance cases in the Council of Europe, so in this European Court of Human Rights, are always against the UK government. It's super interesting, but every like six or seven years, there's a big surveillance case against the UK government. And why is this the case? That's because the UK government, whenever you have surveillance complaint against the government, they have created this funky tribunal which has secret court proceedings. It's a bit like the FISA court and the European Court of Human Rights has said that this is not an effective means to get your rights. It's not effective like a couple of weeks ago because they don't see this tribunal as an effective remedy. There's court cases in several other countries, including in the Netherlands, but in the Netherlands, you have to go through all the lower courts and up, up, up, up, up, up, and in about six years, you'll finally get the European Court of Human Rights to decide something. But it's always good to start legal proceedings in England because you'll have a fast way to the European Court of Human Rights. There we are. This one? Oh, this is Ukraine. Isn't it? Anyway, so there's a lot of action here with the European Court of Human Rights. And I would say, you know, in the United States, if we look at the conclusion of this court case I mentioned before, Judge Smith says that there's a binding Fourth Amendment precedent for video surveillance. This is when, you know, your webcam gets turned on. So there might even be, you know, some traces of constitutional protection here in the United States. But of course, a very dominant school in legal theory is this originalist school which interprets the Constitution like the framers meant it. And the framers were, of course, so visionary that they could envision government hacking. Yeah. Because you said that they didn't know who was the guy? Yeah, so they requested a whole bunch of things. It's in the court ruling. It's in the request, but for example, they wanted to know of this particular device to which other device servers it may contact. So, you know, basically all the internet traffic originating in this device. Proportedly, from this device, this bank account of somebody else had been accessed. But as Judge Smith said, you know, this could just as well be a computer in a public library, you know. I mean, everybody could be behind this device. So that's what they wanted to know. They wanted to... Let's see, I read it yesterday. But the IP addresses was the central thing. I tried to do the identification through IP addresses, which is, of course, you know, another contentious issue. Anyway, it's online, and I think I even have it with me. I have a blackout at the moment. Anyway, to conclude, this right to IT security, I think if we look at a bit longer term, so wiretapping, you know, it started with Olmsted and Cots and all these, you know, rates of jurisprudence. But now, and in Europe, we have like 20, 30, 40 years of jurisprudence before we really got a solid wiretapping jurisprudence in there. I think this constitutional right to IT security is definitely something worth thinking about. And government hacking will only increase as we move further into the IT environment. So that's why I just want to end with raising a couple of questions for research policy and activists. And, you know, feel free to chip in and to discuss, and you know, this is really a sort of open-ended kind of brainstorm here. There's very little data I already mentioned about the root causes of cybercrime. So if we look at game theory, for example, and this wonderful, wonderful paper where do all the attacks go, if you look at game theory, there's actually a lot of you know, cybercrime is a business. It's basically cybercrime is not really attractive to do when you look at, you know, when you try to hack into particular systems. No, what you want to exploit is large scale vulnerabilities. And they capture this thought in this beautiful quote, many attacks cannot be made profitable even when many profitable targets exist. So to tackle cybercrime, what we need to do is instead of, you know, going behind the bad guys, we really need to focus on IT security. And especially in large scale ecosystems, the move very recently of Microsoft not longer to support 400 million Windows XP machines all across the world is something that the legislator should simply not tolerate. Okay, then closer look at the human right to IT security. What about companies? Access the machines all the time, right? So how would this right look at when we look at companies? There's of course the Google ecosystem. Google does, by the way, a pretty good job at IT security. But, you know, it accesses systems all the time. I also want to mention KPN. In the Netherlands we have a net neutrality legislation. And it really came about when KPN very recently at an investor's meeting in London, this Dutch telecommunications company comparable with AT&T announced that it was doing deep packet inspection on all its consumers to know exactly what they were doing all of the time and that they really had a clear picture of what people were doing with their mobile internet connections. Related to that is the form case where form is a deep packet inspection and marketing research company that British telecom in the UK contracted to basically spy on all their users and tell them what they were up to for marketing reasons. If we take this a step further and we've had discussions about this here lately is Ethereum. Ethereum is based on this Bitcoin protocol and creates these distributed autonomous companies. So how is this going to work with IT security? What if, for example, we distributed autonomous companies through an auction model to perform cyber attacks on a particular part of the world. Really interesting questions. We need to start thinking about it. Then European Court of Human Rights accepted in this case, I versus Finland, a positive human right to IT security. So that means that not only should the government refrain from hacking into system but it should also ensure IT security through a specific legislation. This was in 2008, the final judgment. Nobody really took notice but I'll be working a lot on this particular topic and to think about what is a positive human right to IT security. What does it look like? Yes, yes, yes. This is a European Court of Human Rights decision in a case of health data. So a hospital and an employer insured their systems and the hospital said, well, we weren't on our legal obligation and the European Court said, well I think, yes, Finland, because IT security is such a part of the private life, you also not only need to refrain from hacking into system but you also need to ensure that private enterprises are taking your... Positive rate in a similar way. Yes, yes. The European Court. And the German Court was in the law enforcement space and that was a negative right? Exactly. Well, and then if we turn again to a more broader question of intelligence agencies, not so much of law enforcement and companies, I mean, the research questions here are puzzling, right? I mean, basically we learn that intelligence agencies are exploiting every vulnerability in the country and how are we ever going to curtail those operations? There's this very interesting quote, you know, leapfrogging 50 years ahead in this... in a paper that I really recommend you all reading. It's a case note of this German case that considered the Trojan, so the government malware itself as a digital police officer, subject to the same restrictions but also to the powers that its physical counterparts possesses. Think about, for example, the work of Oxford philosopher Nick Bostrom. Think when Trojans get the capabilities to, you know, to look around on a machine and decide, you know, autonomously what are we going to do here? It's very, very important that people in philosophy and ethics already start thinking about artificial intelligence and these kinds of issues. Super interesting. Policy. Okay, cyber crime policy has basically been characterized as this famous quote from Yes, Minister. We must do something, hacking is something, therefore we must do it, right? This is basically not tenable. So we need to demand reliable data for informed policymaking. This is Ross Anderson at all that tried to do this one-half years ago and just one quote, striking example, the botnet behind one-third of the SPAM sent in 2010 earned its owners about three million dollars while worldwide expenditures on SPAM prevention probably exceeded a billion dollars. So think about the mismatch there. We're extremely different inefficient at fighting cyber crime. What should we do instead? Well, catch the crooks basically. Just a week ago, just weeks ago, undercover operation which is an alternative to hacking. You don't really always need to do everything. You can do all sorts of other law enforcement operations and hacking really sounds like you're doing something, but maybe you're not. Another interesting research question is there's a lot about law, policy and technology neutrality. You should craft laws that are technologically neutral because otherwise when the law is adopted it's already not in line with the technological reality. Well, maybe we should think about surveillance law being not technology neutral because new technologies make the surveillance capabilities much more intrusive. So maybe we should actually demand that surveillance law is in some way technology specific and every time you want to expand the capabilities you need a new authorization from the legislator. Research question. Okay, well, again the NSA there's some real oversight needed here and not in a way that law makers only respond when the small satellite is turned at them and thousands of satellites are turned to the outside world. I mean, this is of course an issue that all across the world we need to press our policy makers to do. Game theory again, cyber warfare very different from cyber crime and space publication or this was reported by nature, but game theory really holds very kind of disconferting lessons for cyber war. It's actually having the capabilities is very attractive from a game theoretical perspective. So we will see a lot of escalation here and we have already seen it. It's a bit like cycling. If there's one guy in the entire cycling squad that takes the open and the rest does not he has a huge advantage and I always think about the game theoretical cyber war dimension a bit like cycling and that spurs us to the question about how are we going to curtail this. Okay, so there's both the dimension of countries buying these capabilities from companies and of course there's powerful nations that have the capabilities of their selves. All this, you know, these game theoretical things in the intelligence space really challenge disarmament along with the attribution problem. So people are calling for cyber peace treaties and everything, but it's probably not so realistic. Even though it's needed these insights need to feed into that process and cyber war treaties are really needed along with those trade restrictions, but it will be a lot harder to get them. Finally on activism activists have a short term, very important task here. It's about getting the facts out. Reverse engineer, the malware do trace, report scans whatever, go to conferences at Privacy International Eric King and Chris Aguin at ACLU have been really successful in this going to conferences, obtaining material, trying to get to know what the hell is going on and this bits of freedom in the Netherlands send out open letters to all the antivirus companies demanding software transparency and asking them directly what will you do when the government malware when you detect government malware, well only half of them responded. So the other half, you know we can only speculate. So do a lot of foias on malware try to get these cases unsealed point towards the role of industry and find whistleblowers between in these organizations. In the Netherlands currently under consideration is a hacking law that has way too broad suggestions in them. Think about these 11 problems and advocate against them. Okay, I think this is about it. Final point which has which is a bit broader I'm now here in the United States for like six months and I've seen in the activist community a lot of focus on cyber security on information security and this recent action by access and a couple of other NGOs on encrypt all the things is of course good network security is an important thing but network security is not the same as privacy and this cartoon sort of quite elegantly captures what happens when you as an activist focus too much on security because encrypt all the things actually doesn't mean encrypt all the data on a server it only means the network so companies will be happy to you know sign up for that kind of campaign but at the same time data mine all your data when it's on the server so think about the different values of security and privacy in your vocabulary alright thanks do you have any specific scenarios where people have had their information on cloud data mine? Do you have a Gmail account? It's the business model of the internet so basically every cloud service data mines all your data that you have in the cloud it's not entirely true there are some cloud services that provide some asymmetric key encryption and encrypt your data while it's at rest but yeah you basically have to assume that every cloud company especially when you see ads such as you know basically all the webmail providers they're basically either directly serving you collecting materials or gaining insights from the data they have and selling it to other companies Not like cloud data that's been encrypted or at rest well when data is encrypted it's pretty hard to data mine yeah sure Thank you for a great presentation I'm a little bit concerned about the use of the word hacking when applied to government spying and surveillance I know it's sort of become fashionable I remember the first time I heard it fairly recently I thought I don't like that and the reason is I think it trivializes there's a tendency to trivialize what the government are doing when they're spying on citizens and I think the reason for that for me has something to do with the history of hacking at places like MIT hacking is this sort of harmless thing that people do hackers they're just hacking to respond to that and is there a better language that really captures what's insidious about this it's very interesting I was in a debate about a year ago with some yeah well some high ranked officials in the Netherlands let's put it that way and they were really concerned that the word hacking was part of this debate so you know these guys come from a very different background than you I assume and they think that hacking as part of the vocabulary is really damaging them so that's well hackers come on hackers are dangerous people most people that are not at MIT doing that or around it don't think that hacking is a positive fun thing I think that's the argument have you ever heard of hackathrons so well well but vocabulary is very important and the NSA in its slides talks of sabotage you know kind of a cool word I think but it's really a trojan or malware or whatever I think for activists from an activist perspective the words you use are extremely important I mean that really tilts the debates just as an example in the Netherlands there has been a lot of success with the campaign against electronic voting yeah sure so in the Netherlands there's been a lot of success in the campaign against insecure electronic voting and here in the United States less and when I was talking to Ed Felton at Princeton who is my supervisor there he said that well one of the critical public debate electronic voting machines has been less successful in the States he said is that in the US we call them electronic voting machines and a machine is like a coffee machine you push on a button and coffee comes out whereas in the Netherlands it's a computer you know a computer they spit purple vomit all the time it goes wrong and so people in the Netherlands really the general public understood the problem of a voting computer rather than a voting machine and those are his words and I fully agree with that any cases where the government has compromised the system to gather evidence and then defendant has challenged the admissibility of that evidence based on one of these constitutional theories well I haven't seen them I think that the Dredpire Roberts case is definitely bound to go that way in the United States law enforcement hacking has been mostly sealed so that really also has implications for defendants in a case we haven't really seen it but it's definitely a good question and it's definitely something we will see in the coming well months, years, whatever sure part of the wider consideration much of cyber security kind of strikes me as petty bourgeois like a classical left it's all about keeping your own personal property secure and especially when Ben Bernanke was in congressional testimony he doesn't have to reveal where they sent trillions of dollars just the whole the idea of nationalizing the major IT companies as long as those are private and big data is seen as legitimate it almost seems like we'll never win the concentrated power of corporations being behind governments very good question and I think a lot of the response posed already before the Snowden revelations when we did our research on the Foreign Intelligence Surveillance Act from European perspective and presented this in Europe, like in Brussels the immediate response of policy makers was European Cloud nationalize or localize the interest to enable spying by your own intelligence agencies but not so much by transnational interest so I think that a lot of the people in internet research are still very much thinking in this global perspective but the way in which hacking and surveillance is done is very transnational and the way in which we will see a response from like industry is to further nationalize these kinds of systems not so much the interconnection stuff so I think the Balkanization frame is way overblown but definitely we have seen this for example in routing where the US Congress has said well we don't longer allow Chinese routers in our country like Huawei you cannot do business here or at least not do critical infrastructures for fear of backdoors well of course you know American routers or other routers might have backdoors of themselves which is still an issue open for debate but the question of nationalism and sort of nationalizing the cyber security debate and the critical infrastructure and national security is definitely something that needs a lot more thought and research I completely agree okay sure do you invite the activists to do reverse engineering on these tools as a way of fighting the problem I'm guessing what legal implications could that have given that these are tools from the government to do what's supposed to be law enforcement tasks yeah very good question so let me restate that in itself you know addressing cyber crime is a raison d'etre of law enforcement right it's a really important space to be in but why is the reverse engineering so critical it's just to you know push back to give the right checks and balances so in Germany and I don't know if it's enforced in Germany already at least in the Netherlands I know that there is a responsible disclosure code adopted by the Dutch government so security researchers in the Netherlands when they do responsible disclosure when they for example find government malware in the wild and reverse engineer it and go to the law enforcement agencies and say you know you really should take into considerations these security bugs that frees them from prosecution basically so responsible disclosure is definitely something that also needs more attention and I think that you know if you I think that the environment the policy environment is quite maybe not in the United States I can't speak for that situation but in Europe the environment is changing and really seeing the value of ethical hacking as a security tool so I think that responsible disclosure we will see more of that at least in Europe and then there was one more question So on your slide you had Qualcomm Bot which is the NSA program of hijacking bot ants and it says they have 140,000 bots co-opted and so I'm curious what responsibility do nation states have towards say co-opt a criminal bot what can they do with that do they have more legal power to engage in criminal activities because they did not spread the malware themselves what is the legal framework that they're operating on very good question I think that will be one of the questions to really to work on since last week especially here in the United States so in the Netherlands with the great botnet case they actually went into the botnet, advertised this a successful approach this is a great anecdote so they did this on television so they got the Dutch 8 o'clock news to come to Lee's Web which is this large web hosting company and they set up a fake server and you know a plug and then on the 8 o'clock news they unplugged the botnet it's really bad anyway you know this costs for a lot of pushback from research communities policy and especially activist communities the legal framework here is very opaque and as often in law laws are created or left opaque for certain reasons and that's why you need research and policy and activism to pushback because if we hadn't had this slide over here we would probably be guessing but now we have a really sharp question to ask to our legislators and I think that's your question is spot on and it's really something to work on the legal framework here is especially in the United States when it comes to national security you can do a lot alright, time's up, thanks