 Good morning. Welcome to today's conference. Hashtag Cyberspace IRL. Rule of law approaches to virtual threats. And welcome to the United States Institute of Peace. I'm David Yong. I'm USIP's Vice President for Applied Conflict Transformation. Our Center for Applied Conflict Transformation tests innovative approaches to peace building. As many of you may know, USIP was founded by Congress in 1984 as a nonpartisan national institute dedicated to the proposition that a world without violent conflict is possible, practical, and essential to US and global security. USIP pursues this vision by working on the ground with local partners, local peace builders. We provide governments and civil society organizations with the tools, knowledge, and training to manage conflict so it doesn't become violent and to resolve it when it does. For many decades I've had the honor to work at the intersection of peace, human rights, democratic governance, and the rule of law. So I'm particularly pleased this morning to welcome all of you to discuss the role of the rule of law in the face of virtual threats globally. We are particularly pleased to be co-hosting this important event today with the American Bar Association's Rule of Law Initiative. I'd like to thank my USIP colleague, Philippe Laroumartin, and his colleague, Chelsea Dreyer, for working with our ABA colleagues on this important event. I'd also like to thank Alberto Mora and Judge Margaret McEwen for their great leadership of the ABA Rolly Project. For over a quarter century, ABA's Rule of Law Initiative has promoted the rule of law and democratic governance around the world. Rule of law approaches to virtual threats. Of course, this is a timely event and an important topic. Anyone working on managing conflict today cannot do so effectively without truly understanding the ways in which technology can either support or hinder peacebuilding. It's clear that technology presents new opportunities for peacebuilding. Many peacebuilders have indeed successfully used social media to coordinate demonstrations against oppressive governments. Other peacebuilders have developed software applications to expose corruption and hold governments more accountable. And blockchain technology offers new opportunities to instill greater transparency in the delivery of essential public services or humanitarian assistance to populations in crisis. Yet, at the same time, technological platforms are also being used to promote violent extremism, to spread misinformation, and to undermine electoral processes, often significantly destabilizing fragile or conflict-affected societies. In addition, several powerful countries are integrating particularly invasive technologies such as facial recognition to manage security risks at home and are exporting and promoting these new models of intrusive governance. These models are increasingly at odds with human rights standards. And many illegal trafficking networks, often deeply embedded in local conflicts, have been strengthened also by technology. Therefore, peacebuilding organizations like ours, like the United States Institute of Peace, have a significant interest in understanding the various ways and technology impacts violent conflict. That's why we're particularly excited to be working with the ABA Rowley today. We're happy to be partnering on this important effort with them. The law, domestic and international law alike, is an essential tool to help preserve the opportunities offered by technology. And the law is also an essential tool to protect societies from these risks. Today you'll hear from many experts from governments, from the private sector, from civil society, who are expert on these virtual threats. They'll discuss with you issues like protecting data security, preserving online freedom, countering hate speech on social media, deploying technology to combat human trafficking, and most generally to set more effective legal norms for cyberspace. So with all these experts in-house, including all of you in the audience, I invite you to make new networks in our struggle against these virtual threats. Through these connections, we'll advance our combined efforts. And I offer to you the strong partnership between the United States Institute of Peace and the American Bar Association's Rule of Law Initiative. It's a good example of partnership in this common endeavor. So now I'd like to welcome to the podium Mr. Alberto Mora. He is ABA Rowley's executive director. He's one of our community's great champions for human rights and the rule of law. Welcome, Alberto. Thank you for your leadership. Good morning, all, and thank you, David, for the introduction. And thank you for partnering with the American Bar Association Rule of Law Initiative in sponsoring today's conference. I'm particularly pleased to be here in the Frank Carlucci Auditorium, by the way, since when I graduated from college 45 years ago, Frank was my first boss at the U.S. Embassy in Portugal and remains actually very much one of my mentors over the decades. And not to get on a somber note, but to touch on a somber note, those of us in the American Bar Association family learned of the death of Sandy Dallenbert, who was also a mentor to many of us, an example and a great leader and a giant in American law and in the American Bar Association. So it's a somber note, but Sandy would be delighted that we're holding this conference today on this particular topic and continuing the work that he led over so many years. I'm delighted to open this conference on behalf of the American Bar Association and my role as Executive Director for Global Programs and Director of the Rule of Law Initiative. Since its inception in 1878, the American Bar Association has been partnering with governments and non-governmental organizations, not only to support the development of the American legal community, but also to advance human dignity through the promotion of the rule of law. This conference is the latest manifestation of the ABA's history of convening some of the finest thinkers to tackle some of the most complex and vexing problems we face in the United States and around the world. And the problems we are discussing in this conference are indeed in that category. Every day, we're reminded of the pervasive threats in cyberspace and how bad actors online have an impact IRL. As the young people in our office say, which of course, as I just learned a couple of days ago, stands for in real life, or so they say. So I'd like to welcome everybody to this year's American Bar Association's Rule of Law annual conference on contemporary rule of law issues entitled hashtag cyberspaceRL rule of law approaches to virtual threats. This is our third annual issues conference, a time each year which we take a step back from the day-to-day implementation of rule of law development program in more than 50 countries to reflect on that work, to learn on what we're learning, and to convene experts from whom we can learn and together strengthen the rule of law and governance communities. We're delighted that you're here to help us in this work, and let me say a few words of acknowledgement to the many people who have helped make this conference possible. First, to set the stage for this work, we commissioned a paper on the topic in ABA Roli's paper series, and I want to commend co-authors Mary Greer of the ABA Roli Research Evaluation and Learning Division, and Tara Moborowski of ABA Roli. They've done a masterful job of condensing a survey of series of difficult, interrelated issues. And second, I'd like to recognize Dr. Linda Beshai, ABA's Director of Research, who has led this effort from the beginning and will be working to see that we're learning from this conference and that learning is captured and incorporated in our future work. With the help of Raptors Notes and under Linda's direction, we will compress the learning from today into a conference report and recommendations will be posted to our website along with a video of each of today's panels. Linda, if you would please stand up with members of our Artreach and Conference Working Group and stand. These are the folks who have made this conference possible. I'd also like to recognize the ABA Roli Board of Directors, led by our Chair, Judge Margaret McEwen, and its Program Committee led by Judge Jamie Baker. Their insights, advice, and comments were invaluable in helping this initiative and we're honored that so many of them could be with us today. We also have the privilege of having with us the President of the American Bar Association, Bob Carlson, and I think Judy Paris Martinez, the President-Elect, is also in the audience who will be coming shortly. Their presence is evidence of the importance that the ABA places on this topic and our work to contribute to the rule of law approach to this debate. Let me also say a big thanks to the dozens of ABA Roli volunteers, staff interns who also participated in making this conference possible. Finally, let me thank the ABA cosponsors and their members, ABA Standing Committee on the Law and National Security, and the A Criminal Justice section. And with that, let me invite the Chair of the Roli Board, Ninth Circuit Judge Margaret McEwen, to the podium to say a few words and introduce our keynote speaker. Thank you also very much. Good morning and thank you, Alberto. On behalf of the ABA Rule of Law Board of Directors, I also thank all of you for coming to join in this conversation today. One of the strengths of the ABA over the years has been convening different stakeholders, and that's what we've done today. We have here, if you look around and meet your neighbors, everyone from international organizations and law enforcement. We have lawyers and non-lawyers, private sector, donors, implementers, academics, and practitioners. So our goal today is to take a look at these complex issues, but to do it through a rule of law lens. So here's the question that we will pose today and that our speakers will in various ways be addressing. That is, what do rule of law frameworks and interventions offer us in terms of grappling with some of these cyber complexities? You all know that in general, virtual threats are often dealt with in the framework of security. What we want to look at today is ways in which rule of law approaches are really at the center of dealing with these issues, whether it's in terms of strengthened frameworks for either defining crimes or harnessing global cooperation, or in terms of an architecture of how rule of law intersects with these cyber issues, looking at ways to have protection of both civil and political rights, freedom on the internet. So we think the rule of law community has a very different and very positive perspective to offer on this. Obviously today these issues of cyber are front and center, but I'd like to put them in context in terms of a temporal scope. In the early 1990s, with the fall of the Soviet Union, the ABA rule of law initiative was born, first in Eastern Europe, and then we spread to all of the continents. So that's the early 1990s. Recall that the internet was not in common currency until probably the mid 1990s. And the first time our U.S. Supreme Court dealt with the internet was not until 1997 in the case of ACLU v Reno related to the online protection of children on the internet. That was 1977, and of course since then the floodgates have just opened and we've gone from there. So I think we have a lot to learn from each other today, and I thank you in advance for this important conversation. Very fortunate for us, we're extremely pleased to welcome a prominent voice in this arena to be our keynote and kickoff speaker. That's Sujit Raman. He is currently the Associate Deputy Attorney General in the U.S. Department of Justice. Here he is advising not only the Attorney General, but many others in their oversight of our country's cyber-related criminal and national security issues. He's also before that was a line prosecutor, so he's seen it from the bottom all the way to the top in terms of his job. And he also represents the Department on Cyber Matters both before the National Security Council at the White House and other agencies in the U.S. government. So he has a very unique perspective to offer us, and I now thank you for coming and ask Sujit to please join us and welcome him. Thank you. Good morning. Truly is a privilege for me to be here this morning, and I am honored to follow to the podium three prominent defenders of the rule of law, both at home and abroad. Thank you, Judge McEwen, for those kind words of introduction. Thank you, Mr. Mora and Dr. Young, for your comments as well, and for dedicating your lives to upholding and promoting the ideal of the rule of law. Our theme today is the rule of law in cyberspace. No topic could be more timely, and no domain poses a greater threat to America's public safety and national security. As the Director of National Intelligence observed earlier this year, our adversaries and strategic competitors will increasingly use cyber capabilities, including cyber espionage, attack, and influence to seek political, economic, and military advantage over the United States and its allies and partners. The threat isn't limited to state actors. FBI Director Chris Wray informed Congress last fall that we also face sophisticated cyber threats from hackers for hire, organized crime syndicates, and terrorists. Not only do these threat actors constantly seek to access and steal our nation's classified information, trade secrets, and technology, they also seek to strike our critical infrastructure and to harm our economy. Reading quotes like this, you might think cyberspace is the 21st century equivalent of the 19th century Wild West. It's true that actors in ungoverned physical spaces aim to spread instability around the globe, including through cyber means. But much of the world's cyber instability today is caused by actors who live in tightly governed spaces, authoritarian nations like China, Russia, Iran, and North Korea. These nations actively work to destabilize the U.S.-led international order, thereby promoting and advancing their own geopolitical interests. I'd like to use my time with you this morning to discuss three interrelated ideas. The first is that we have moved into a new era of great power competition. In its early years, the Internet's openness seemed to represent an unvarnished good that would promote free thinking and human rights around the world, defeat authoritarianism, and drive the growth of new markets. Today, that vision unfortunately has darkened. Our adversaries have used the Internet to exercise control over their own populations, and they've leveraged cyberspaces open and unregulated character to threaten our economic and national security, narrow our nation's strategic advantage, and attempt to undermine our values. The second idea relates to the importance of law in the new age of great power competition. International law, that is, the law governing the relationships between sovereign states, applies in cyberspace and could serve as a stabilizing force in this new era. But international law in this context is still in its infancy, and many of the details remain to be worked out. The reality is that states often tend to follow their own rules, especially in the gray zone between peace and war, where so much of today's great power competition is taking place. For that reason, I will emphasize U.S. domestic law and the important role that the U.S. Department of Justice in particular plays in defending the rule of law in cyberspace and in promoting global cyber norms. Finally, I'll briefly address a characteristic aspect of the new era in great power competition. That is, foreign adversaries' weaponization of information in their attempts to sow discord in our society, undermine our democratic values, and disrupt the rule of law within our own borders. I'll explain how our domestic legal traditions empower us in this brave new era. And I'll leave you with this thought. Under the Constitution, we are not powerless to confront and to counter other nations' covert, deceptive misuse of cyberspace in their efforts to turn our free speech ideals against us. Now, cyberspace was once seen as a medium for accelerating the spread of liberal values. As Ira Magaziner, President Clinton's internet czar, observed in 1999, the internet is a force for the promotion of democracy because dictatorship depends upon the control of the flow of information. The internet makes this control much more difficult in the short run, he wrote, and impossible in the long run. In addition, Magaziner stated, the internet will promote better understanding among nations and will be a tremendous force for improving education. Many thoughtful people shared those views, which in one form or another animated the internet governance policies of the Clinton, Bush, and Obama administrations. Sure, technology firms experienced tremendous growth in those years, and the networked, interconnected world did draw closer together, creating obvious benefits. But since then, we've learned that many of the policy choices made in the internet's infancy sowed the seeds of that early vision's destruction. To understand why, it's useful to consider the internet's structure. The way our defense strategists sometimes conceive of it, cyberspace is composed of three interdependent layers, physical, logical, and persona. Hardware and tangible infrastructure comprise the physical layer. The human beings who develop a digital representation of themselves, such as an email address or a social media profile, to participate in interconnected activity comprise the persona layer. Digital is the logical layer. In essence, the internet's nervous system. Colloquially, we think of this layer as code. This is where data exists. It's also what links network nodes together as packets of information travel around the world. To promote rapid growth and universal connectivity, those who shaped the internet prioritized the logical layer's openness, speed, and mutability. They did not prioritize its security. Many internet protocols simply assumed that every other computer on the network could be trusted. Thus, for all of its benefits, this open structure also came with a cost. It wasn't long before our adversaries began using this aspect of the internet's openness against us. At the same time, the internet has never been truly borderless in the sense of being completely undifferentiated. As Jack Goldsmith observes, the very notion of an internet implies fragmentation. As data packets move from one connected network to another, they can be filtered and routed based on various attributes, including the source of the data and its destination. Authoritarian nations have used filtering to control the content of the data that enters and exits their borders and to monitor and censor what moves around within them. China's Great Firewall is probably the most notorious example. Russia's reported plans to unplug from the global internet fall into the same category, as do that nation's repeated calls in international fora for greater information security. The growth of the internet actually fortified illiberal regimes hold over their citizens and empowered those governments in new and alarming ways. Taken together, these trends have fundamentally pre-oriented international relations. Over the past few years, our competitors have used cyberspace to consolidate authority at home while accessing our information and intruding into our affairs in ways they could never do in a purely physical world. All the while, they have carefully sought to operate below the threshold of the use of force, so as not to trigger an armed U.S. response. Nevertheless, their tactics have increasingly become assertive. As General Paul Nakasone, the commander of U.S. Cyber Command, observes, 10 years ago, cyber threats were primarily about other nations engaging in espionage by coming into our networks and stealing information. That activity was serious enough. The situation turned for the worse, starting in 2013, when our adversaries began disrupting a series of networks within the United States, often targeting and victimizing private parties. From the denial of service attacks that Iran launched against our financial sector to the destructive attack against an American casino, from North Korea's targeted attack on Sony Pictures to its reckless propagation worldwide of the WannaPri ransomware, from Russia's spread of the destructive not-petia malware to the information operations it has launched against its adversaries around the world, and from China's mass theft of U.S. government personnel data to that nation's sustained campaign of intellectual property theft. Our adversaries, in General Nakasone's words, have mounted continuous, nonviolent cyber operations that produce cumulative, strategic impacts by eroding U.S. military, economic, and political power without reaching a threshold that triggers an armed response. This, in a nutshell, is what we mean when we say that great power competition has revived and that its locus has shifted to cyberspace. Both the latest national security strategy and the national cyber strategy identify a number of priority actions to ensure that America remains safe in this new era. Law enforcement plays an important role in our shared, all-of-government effort. At the U.S. Department of Justice, our primary missions include forcing federal criminal law and protecting national security, combating cyber crime and cyber-enabled threats to our nation rate among our highest priorities. For many years, we've targeted and successfully disrupted transnational criminal syndicates engaged in cyber crime, as well as the digital infrastructure those actors employ. More recently, we began publicly charging foreign state actors whose malicious cyber activity broke U.S. law. It's fair to ask why we devote significant resources to prosecuting state actors whom we may never bring to the United States to face justice. And it's equally fair to ask why we shifted from an approach that relied mainly on intelligence collection and diplomacy to one that includes a law enforcement response. As I will explain, the contributions of state-sponsored malicious cyber activity serve an important purpose. Even if we can't guarantee that we'll be able to produce in court every individual involved. Since the indictment of Chinese PLA officers in 2014, the Department of Justice has remained focused on state-sponsored criminal activity that targets U.S. companies. We are also now focused on activity that targets the U.S. criminal process. In the past two years, the Department brought more national security cyber cases against criminals acting on behalf of our major adversaries than in the previous five years. There are several reasons for the increasing prosecutions. The main one is that we are following the threat, just as we did in responding to the threat of terrorism. As I have explained, nation states are engaged in activity and companies in the United States violates U.S. law and departs from international norms of responsible state behavior. Norms that benefit all nations. Our criminal cases reflect our adversaries' efforts to harm our companies and our nation. Second, the increasing number of national security cyber cases reinforces the lesson that our adversaries' conduct lies outside the norms of responsible behavior. The actions we highlight in indictments are not legitimate statecraft. They are crimes without justification in international relations. I'll say more about that in a moment. Third, our cases reflect our increasingly sophisticated ability to attribute this criminal conduct to the individuals and states involved. This ability is closely related to my second point because it shows the commitment of our law enforcement and intelligence agencies to work closely together while protecting intelligence sources and methods. These partnerships, which were forged in the counter-terrorism context, serve to solidify the consensus that a law enforcement response to malign nation-state cyber activity makes sense. In bringing these cases, these basic principles. First, the department has a duty to enforce our laws and protect our people. We cannot refuse to act when foreign state actors violate our criminal laws, transgress, establish norms, and victimize our citizens. That is especially true when such crimes often represent severe violations of the victim's privacy rights and can have lasting, damaging consequences. The department has an obligation to work toward a future where our citizens can live and conduct their business with confidence in the integrity of their information and institutions. Second, attribution is the key to deterrence. Without attribution, there will be no consequences and thus no deterrence. Attribution through the criminal justice system escalates the stakes that a police or a public statement alone would not. We have on occasion obtained custody of foreign criminal defendants. Our indictments limit their travel and the prospect of criminal indictment can help deter some cyber actors from engaging in such conduct in the first place. This can make it more difficult for states to recruit the manpower and resources for cyber attacks and raise the cost of engaging in a malicious cyber activity. Third, attribution through the criminal justice system is a powerful way to expose state conduct that violates norms of responsible behavior. It complicates our adversaries' attempts to feign ignorance of illegal acts they thought could be taken in secret or to hide behind public denials. Our cases are governed by well-known policies relating to the conduct of all federal prosecutors. An indictment is brought by a grand jury under established procedures. Chargers are brought only when the facts and law justify it. The allegations in our indictments are thorough and detailed and we can prove them in a courtroom using admissible evidence at proof beyond a reasonable doubt. For all these reasons, criminal indictments are among the most powerful statements we can make as a government. Fourth, unsealed indictments promote transparency. There will always be cases in which our ability to expose malicious cyber activity is limited by our obligation to protect intelligence sources and methods for sensitive ongoing investigations. But where we are able to do so, we strive to expose such schemes to the American people and to the international community. Attribution through detailed indictments educates the public about our adversaries efforts and methods to spread disinformation, steal commercial technology and target computer networks. Fifth, although our goal is to hold accountable in court those we charge with trade theft or cyber crimes, our investigations can provide critical support for the use of civil, diplomatic, economic and military tools. Some thoughtful critics have criticized the department's so-called name and shame strategy on the theory that our indictments have failed to staunch the activity. But you can't separate our indictments from the broader array of tools our government now uses to counter malign cyber activity. These include freezing assets or prohibiting transactions or adding companies to the Department of Commerce entity list. As the National Security Advisor has confirmed, it also includes undertaking offensive cyber operations aimed at defending our national interests. Our tools also include other authorities that can block criminals' assets, restrict their access to the banking system and prohibit them from freely engaging in trade. We developed this approach to address terrorism and terrorist financing. We are applying it to the cyber context as well. Finally, by using public law to emphasize the need to protect private U.S. companies and victims against nation-state actors, we help develop the framework for public-private cooperation that is critical to cyber security. The department tries to show through our actions how we can help companies respond to nation-state threats they cannot face alone in a way that respects their status as victims. The department has developed strong relationships with the private sector based on our aggressive pursuit of criminal nation-state conduct ranging from cyber theft to information operations using third-party social media platforms. Now, no one suggests seriously that we can prosecute our way out of this problem but to dismiss the role that federal law enforcement plays in the government's shared fight against cyber-enabled threats is to unfairly discount and diminish our nation's powerful commitment to the rule of law both within our borders and without. Before I conclude, I'd like to briefly address one final topic. That is the question of how we can respond to one of the latest and most potentially destabilizing manifestations of great power competition in cyberspace. Namely, our adversaries use of covert information operations to influence and subvert our nation's democratic institutions, including specifically our elections. The Department of Justice has been instrumental in revealing that foreign actors create and operate false U.S. personas on internet sites designed to attract U.S. citizens to spread divisive messages. They also fabricate news stories in an effort to discredit American individuals or organizations. In the process, they reach unprecedented numbers of Americans covertly without ever setting foot on U.S. soil. These deceitful actions are especially pernicious because they seek to weaponize our traditions of free speech, open inquiry and individual conscience against us as part of a broader project to undermine the very concept of self-government. Foreign attempts to pollute our public discourse are nothing new. These efforts have taken many forms across the decades from covert funding of newspapers and covert financing of front groups to creating and spreading fake internal government communications. Our traditional response has been one to require transparency. The Foreign Agents Registration Act for FARA, for example, requires persons who engage in certain conduct as agents of foreign principles to register with the Justice Department, to file periodic reports thereafter and to include a conspicuous statement disclosing that relationship on any materials disseminated by the agent on behalf of the foreign principal. FARA's purpose is to ensure that the American public and our lawmakers know the source of information that is provided at the behest of a foreign principal where that information may be intended to influence U.S. public opinion, policy and laws. The statute enhances the public's and the government's ability to evaluate such information. Our recent indictments exposing Russian malign influence activity fall within the same heritage. Uncovering and disclosing such malign activity after it has happened is not a panacea, however, especially where public disclosure of a foreign influence operation could amplify it or could create undue public harm or confusion or could compromise intelligence sources and methods. That raises obvious questions. In defending our elections, are we limited to enforcing federal disclosure laws and other federal criminal laws that address foreign interference in our elections? And are we limited to uncovering and disclosing such conduct only after it occurs? Or can we take affirmative action to prevent covert cyber-enabled foreign influence campaigns that are designed to attack and undermine our elections through the weaponization of speech? I think the answer to that last question is yes. If non-US persons outside the United States are covertly interfering in our elections, whether through malicious cyber-activity or covert operations using social media, the government can act to prevent that conduct consistent with the First Amendment. The fact is, it has been settled law for over a century that non-US persons located outside the United States have no rights under the First Amendment. Foreign governments similarly lack First Amendment protection. The more difficult question is whether hypothetical US government efforts outside of our borders to block or target such activity would impact the First Amendment rights of US persons who are potential consumers of the covert foreign dissemination. Or for that matter, the free speech rights of US persons whose own online speech may be amplified by covert foreign activity, as where covert foreign controlled accounts disseminate the content using bots. In a line of cases stretching back several decades, the Supreme Court has indicated that the First Amendment encompasses a right to receive information. That is, a right of a would-be recipient of information that is independent of any right possessed by the speaker. Hypothetical activity by the US government to prevent foreign dissemination of information to the American public could, therefore, implicate the First Amendment rights of US citizens and residents to receive information. In my view, however, such hypothetical activity would likely not violate the First Amendment where it targets messaging covertly disseminated by a foreign government and or its agents seeking to interfere with the US election and where our government's actions are based exclusively on the foreign source of the information. That is because the Supreme Court precedent suggesting that Americans might have a right to receive foreign political propaganda and circumscribing the government's ability to limit the stock of information from which members of the public may draw decisions presuppose that the recipient can weigh the information he receives in light of the source of that information so as to evaluate the import of the propaganda. The calculus is very different where the foreign actors have deceitfully misattributed information in a manner designed to mislead rather than to inform. The First Amendment does not provide a right to receive covert foreign propaganda. Otherwise, disclosure statutes like FERA would be unconstitutional. Hypothetical US government activity would likely be consistent with the First Amendment even where it involved action to prevent foreign governments and their agents from covertly amplifying the online speech of Americans. The speech of Americans is of course fully protected by the First Amendment. The government thus could not remove or repeat the online communications of a US person. But such a person has no constitutional right to amplification by a foreign government which itself, of course, is without constitutional rights. The Department of Justice has previously expressed the view that a US person has no First Amendment right to speak on behalf of a foreign nation. On the theory that the US person speech in that context isn't his speech at all that of his foreign master's unprotected voice. In much the same way, a US person's ability to speak is not impaired by the denial of amplification from a foreign nation that lacks First Amendment rights. At least where the amplification conceals the role of the foreign nation. I should emphasize that my thinking here is tentative and the context is hypothetical. Moreover, my thinking assumes two important factors. Namely first, hypothetical US government activity would target covert acts by a foreign government and or its agents. And second, our activity would focus on protecting the integrity of the US electoral system. Though the analysis for covert foreign speech relating to political issues generally may well follow a similar track. Under these circumstances US government activity to regulate harm from foreign misinformation would be on the most solid ground. So let me close by emphasizing that how we respond to the challenges posed by this new era of great power competition in cyberspace will have far reaching consequences. As our adversaries use fraud, theft and deception to project their power and undermine internationally supported norms, I'm reminded of the words of US Supreme Court Justice Robert Jackson. Writing in the wake of the Second World War when the global community faced a different set of emerging challenges Justice Jackson noted that quote, we are put under a heavy responsibility to see that our behavior during this unsettled period will direct the world's thought toward a firmer enforcement of the laws. I can think of no more poetic a description of our duty today. We too face an uncertain future and we too must act in accordance with law ensuring that everything we do in these unsettled times directs the world's thoughts toward a firmer enforcement of the laws. Thank you to every single one of you for everything you do to aid in that noble effort. Thank you very much. Thank you Mr. Rahman that was a terrific, terrific beginning. He has so nicely framed how the internet challenges our traditional notions of jurisdiction international relations security intellectual property privacy, speech and the elections. You know it was once said that cyberspace is not a place and the word cyberspace actually comes to us from science fiction but I think as Mr. Rahman has so ably demonstrated that it is no longer science fiction but it is a reality or as Alberto said in real life. So we now move to the second phase of our program sessions. One on the legal frameworks both international and regional in terms of cyber crime and internet regulation and a second referenced by Mr. Rahman on election security. So these rooms are listed in your program but they're both across the atrium from where you had your coffee this morning and you have a chance to choose between either of these and we'll have more sessions as we go through the day and again you can choose or you can wander in and between the breakout sessions whatever your attention span may be. So again please join me in thanking Mr. Rahman for kicking off the conference. We have a lot of ABA and USIP staff here to help you and direct you enjoy the day. Any of these gatherings. Good morning everybody ABA and to USIP for organizing this panel and for having me participate as moderator. We have a very distinguished group here this morning to discuss international legal frameworks and capacity building related to cyber crime. Before I begin I wanted to introduce myself and give a very brief context and then we'll turn to our panelists. I work in the State Department's Bureau of International Narcotics and Law Affairs where I'm the acting director of the Office of Global Crime Issues. For those of you who haven't heard of INL we help our foreign partners tackle the most pressing international crime issues, illegal drugs law enforcement that impact the United States and its interests. We have capacity building programs in about 90 countries around the world and our partners to implement those programs include US criminal justice agencies like the US Department of Justice international organization contractors non-governmental organizations the American Bar Association among them and with our help the idea is that countries are better able to confront to address crime and to work with us to address crime before it impacts our citizens. Our classic mission international narcotics and law enforcement our classic mission has been counter narcotics and that still is a critical area of work for us especially in light of the opioid crisis. But we've also seen our programs evolve over the years to meet emerging threats and of course it goes without saying in the setting countries around the world developing and developed have seen tremendous challenges facing the criminal misuse of information and communications technology and the crimes of course go from ransomware to botnets to cyber enabled fraud and online child sexual abuse. While strong national laws and capacity are important to combat this threat of course cyber crime is the quintessential transnational crime and we need international cooperation to effectively combat it. International legal cooperation such as mutual legal assistance is necessary to pursue cases when the impact and the evidence may lie across borders and while every country's law doesn't have to look exactly the same as every other country's some commonality some interoperability helps with cooperation by the same token bringing countries if not to the same level of capacity but trying to bring countries to a similar level of capacity fosters cooperation and makes it more feasible. Now the tools that are out there some of which we'll discuss today range from bilateral agreements like mutual legal assistance treaties informal networks to foster law enforcement to law enforcement cooperation the G7 24-7 network of high tech crime points of contact is one example multilateral instruments we'll talk about the Budapest Convention even the UN Convention on transnational organized crime or UN talk. For the U.S. our cyber crime policy in our programs because we do a lot of capacity building we support a lot of technical assistance globally in this issue as with other issues is to promote accession to the Budapest Convention. At the technical level it outlines and we'll hear more from Rudy and others as he explains the essential elements of a strong domestic counter-cyber crime regime and at the policy level it reflects the values of liberal democracies in terms of promoting free speech a multi-stakeholder approach to to cyberspace and respect for human rights. In contrast and this will be part of our discussion this morning some countries have began advancing alternative or contrasting approaches that stress a more top-down authoritarian approach that may threaten free speech human rights and insert greater state control I would say into cyberspace and the response on specific issues like cyber crime. So we'll talk about what is out there what opportunities are presented by the existing international legal framework what the challenges are and alternative views we'll also talk about partnerships including capacity building partnerships and the important role that they play. Now I'm not going to try and what I hope to do is we'll have relatively brief presentations from each of our panelists and then we'll open it up for comments and questions from the whole room and when I say comments I envision this as participatory and that the only expertise certainly doesn't on this issue certainly doesn't lie here at the front of the room. I will ask that when we have questions or comments just that you introduce yourself say give your name and your affiliation. I'm not going to do lengthy introductions to the panelists because each is sufficiently distinguished that we would use up the next half hour if I went into the full depth of their bios. Take your time, please. Sure. Rudy is not shy. In the order that we'll go though Rudy or Hollis is both a consultant for the Council of Europe and the chairperson of the group of experts on cyber crime for the organization of American States. He was a DOJ senior trial attorney at the computer crime and intellectual property section where he prosecuted many, many, many major cases. He was a legal attaché at the U.S. Embassy in Argentina and an AUSA in San Francisco. Ken Kern all the way to my right is the chief information officer and the special assistant for international relations at the New York County District attorney's office. He was the deputy chief for cyber crime and identity theft there where again he prosecuted a wide variety of high level and high impact cases in identity theft, financial and cyber frauds. He was a prosecutor with the ICTY, the International Criminal Tribunal for Yugoslavia and participated in the Milosevic trial which brings us to Mary Greer who also worked with the ICTY. Mary Greer is the senior technical advisor in criminal law at ABA Roli. Her work with the rule of law initiative then CLE dates back to 1998 when she was a liaison in Bosnia. She designs and implements programs in global criminal justice issues like cyber crime but also sexual and gender-based violence and financial organized crime. So we have a wealth of perspectives. I am now going to silence myself turn to Rudy and then we'll go to Ken and then to Mary. Thank you. I was recently in New York City passed away and I had the responsibility of going through her things to basically close out her state. Do I need the microphone? Yes. And in going through her things it was like going through a museum. One of the things that I found was this. A transistor radio which were developed in 1955 it revolutionized electronic communications like nothing ever before. It sold over a billion. It facilitated people being able to go out and listen to music and the news and it contained four transistors. Today's communications are through devices like this which contain over four billion transistors. Now the problem is that many countries are operating with laws that they enacted and have not changed since 1955. And as you can imagine these laws in these countries are not equipped to deal with the digital world and the realities of the digital world. As an example on May 5th in the year 2000 the I Love You virus was released. It infected over 55 million computers resulting in over $10 billion in losses. The Pentagon had to shut down its servers. The person was identified as Odell Del Guzman who is from the Philippines. Although he was identified he was never prosecuted. He was never imprisoned because at that time the Philippines did not recognize the realities of the digital world. Yes they had laws that dealt with destruction of property but they only dealt with tangible properties, not with property of ones and zeros. And so it wasn't recognized as a crime and extraditions for Mr. De Guzman's arrest and extraditions were obviously denied. At the same time a number of countries in Europe including the United States and Canada foresaw these new challenges that we were all starting to see and they negotiated the terms of an international treaty which is the Budapest Convention. They recognized that cyber crime is an international problem that no country can deal with it by itself. It requires cooperation among various countries. So after a decade a decade of negotiations the Budapest Convention was open for signature on November 23rd 2001 and the treaty was open for signatures. The treaty is basically broken down into three components. Part one deals with four basic definitions. What is a computer system? What is computer data? Who is a service provider? And what is traffic data? Articles 2-13 deal with substantive laws and it basically says no matter what your legal system is we have to agree that certain things are going to be criminalized. For example illegal access to a computer illegal interception computer related forgery computer related fraud child pornography intellectual property corporate liability and the third component has to do with procedural laws and these are basically it says again hey no matter what your legal system is you have to have certain things in place in order to deal with the realities of the digital world. We're operating under the common law system under the Napoleonic system and in some countries it's the judges for example who do investigations and it says no matter what your legal system is we need to do things like have some way to expeditiously preserve data because we recognize that data can disappear by touching a button and we don't have the time to get judges sometimes to sign orders and we need to be able to act quickly so it encourages countries to be able to have a mechanism to preserve data without the need of a court order it also calls for certain safeguards it says yes we have to act quickly but we also have to take into consideration privacy there has to be a balance between privacy of our citizens and the needs of security when you walked into this building you gave some of your privacy when you went through that metal detector when you go to an airport they search your luggage you're giving up a lot of privacy but again balanced against the needs and the realities of security it must be done but the buddha best convention specifically article 15 says countries one over the other you have to have a reasonable balance so as a result the buddha best convention is now has been signed by over 66 countries it is it deals with criminal behavior but also it has been used as a mechanism to save lives how is that through the signatories kidnapped victims have been identified a few years ago I don't know if any of you have heard of something called the blue whale the blue whale is a phenomena where disenfranchised adolescents find each other they encourage each other to mutilate themselves and sometimes even commit suicide I think there have been over 200 kids who have committed suicide until Argentina found that someone in their country was trying to get a group together to do exactly this and they contacted the department of justice because they are also a signatory to the buddha best convention and they asked us to intervene on behalf of them with facebook hoping to identify who these kids were because of the buddha best convention and the various treaties we were able to identify the source and save six lives all of them found in Argentina the treaty is dealing with the new cyber issues as well for example in the year 2016 it published its conclusions and recommendations involving trans-border access to electronic evidence in the cloud and this was the basic issue in the supreme court case last year which was the case of the buddha best versus microsoft and that case as you may know it involved the question is an order issued by a u.s. court enforceable when the u.s. provider has transferred those records outside of the united states that was an issue that a cloud committee has never resolved by the court because last year the united states enacted its cloud act which basically allows these court orders to be respected both when issued by a u.s. court and in some cases by courts outside of the united states but the point is that many of the findings and conclusions made by buddha best have been incorporated into this new act as you can see I'm high tech today so capacity building is also an important component of the buddha best convention we try to do training on a regional basis and we work with other international organizations in this hemisphere we've worked with the organization of american states with the department of justice and with the state department we've trained over 2,000 legislators judges prosecutors investigators and it has been training that has been given a careful thought instead of just organizing training that come if you want we have been very careful in deciding where to hold this training we try to select countries that have shown a true interest and a commitment to this issue not just always going to Buenos Aires but it's going to be a fabulous for steak and wine but countries like Ecuador or Paraguay if I'm not recommending anyone go to Paraguay it's one of the most boring countries but they have shown a commitment to cybercrime and they were worthy of hosting one of these regional trainings the participants are carefully selected we try to identify people who really need this training judges for example come in contact with electronic evidence but don't feel comfortable determining the admissibility of electronic records we try not to select the same attendees I've gone to train in a number of these various conferences and it always seems that it's the same people attending to hear presentations by the same people so it's like a family gathering and that's something that we've been very aware of and we've tried to avoid the trainers are very carefully selected we want trainers who understand the cultural subtleties that are involved who go down to a foreign country and don't start talking about the red socks versus the white socks because people scratch their heads and say socks what does that have to do with anything we want trainers who speak the language a lot is lost through translation especially when we talk about technical issues technical issues we include forensic experts for example in finishing I want to recognize both the department of state particularly INL who without their grants none of this would have been possible none of this they are the most significant contributor to the Budapest and Council of Europe I want to also recognize the Department of Justice who have provided particularly the computer crime and intellectual property section who have provided the needed leadership to not only negotiate the terms of the Budapest Convention but they've also been a source of very qualified instructors and combined with these we've had the type of success and hopefully we'll continue to receive those funds because funds are being tighter and tighter to receive we recognize that but I also want to be on record to thank INL thank you thank you Rudy I think you raise interesting points that we'll explore further about considerations that inform effective capacity building in general in this area in particular and the tie between capacity building initiatives like accession to the Budapest Convention but the organizers did ask me to make clear that Rudy's views on Paraguay are not necessarily shared by the ABA, USIP or the US State Department diplomacy has never been my strong point never having been there I'm sure it's a very interesting country we'll turn over to Ken now good morning everyone on behalf of District Attorney Cyrus R. Vance Jr the New York County District Attorney I just want to thank ABA and USIP for the opportunity to speak with you today and good morning to everyone a little bit about our office we are at its heart a local prosecutor's office oh it's not okay thank you so on behalf of District Attorney Cyrus R. Vance thank you so much for the opportunity to speak with you this morning we are at our heart a local prosecutor's office we handle approximately 60,000 to 100,000 cases a year based on volume of criminal activity and what that really places is that we become an emergency room of society what happens on the streets of Manhattan come into our doors we have to study it analyze it for our work in investigating exonerating and prosecuting at the same time because of our physical location in Manhattan we are also an international office our jurisdiction because of Wall Street and because of some technological jurisdiction issues gives rise to an opportunity for us to view the lens both as a local office and so we as in trying to address the scourge of cyber crime we see it affecting not only our local institutions and our residents but our international partners and international citizens so we've had to think differently and act differently because of that we've invested in personnel we've invested in making sure that we have process all the things that were coming in the door the same thing that if you were focused on DNA in a medical case you need to have the tools and equipment to focus and drive these cases and to educate your staff to be able to have conversations with judges about why these cases matter the other thing we've had to do is really change the way that we place personnel so we have succumbents as a local prosecutor's office in the city of London Paris, Singapore and in Israel because of the threats that we those are central cities for us in trying to combat cyber crime and financial crime the district attorney's general operating principle has always been it is better to prevent a crime than try to address it from a defense of posture so we've had tremendous discussion about in a few minutes about how to go ahead and try to prevent more of the cyber crime that we're seeing and not to just admire the problem or be frustrated by the problem of cyber crime which is where I think many of our citizens are throughout the world you see a hack a story comes up about 150 million people there's a wow factor and you move on in addition to making sure that we don't just admire the problem that we try to prevent it we've had to become more sophisticated international office we share some of the same principles that Rudy articulated about having staff that can speak across borders and to work with colleagues throughout the world because that is the only way you can do cyber crime and focus on it nowadays so when we talk a little bit about the threats of cyber crime what I want to just show to you are three initiatives, two very positive and one that's been troubling for us as an office you can imagine that as law enforcement officials there is a desire to not to bring people to justice who are committing these irrespective of constraints of borders so we abide by of course all the international norms and follow the policies associated with those and work with great federal partners many of whom are in the room today we have also as a local actor been constrained by some of the time sensitive issues related to prosecuting these cases and agility across networks and borders and many of us at the state and local prosecutor side which handles approximately 95% of the cases in the United States alone did not have the ability to react in real time so you're constantly playing defense and you're constantly slow footed in order to change that the hope that the district said is let me try to invest in initiatives that focus on both prevention and an international framework so I'm going to focus on two of them and I have a deck and I'm set to go if you are sir thank you it's not moving ahead cyber criminal but I'm not going to waste time but as you're getting it I'll talk a little bit about it the district attorney called the global cyber alliance created this initiative a non-profit because in terms of playing whack-a-mole with criminal cases we can't prosecute our way through this problem the only logical solution is to try to have continue our enforcement work but also develop skills at prevention so the district attorney with the city of London police commissioner in September of 2015 created an initiative called the global cyber alliance it's an international organization that is it's really predicated on how do we move real-time data across sectors and countries and we came into this in trying to aid the district attorney as to whether this was a worthwhile investment saying doesn't this exist already and of course there are tremendous initiatives that exist throughout the world but for our vantage point we were not necessarily privy to some of these and when we spoke to our New York partners in the cyber professionals in New York they were certainly not aware of some of the opportunities to share across borders and sectors so the global cyber alliance was created a separate non-profit from our office four years later we're very proud that there are 250 partners across 18 sectors in 28 countries the sectors represent the gamut of ones you would think of from critical infrastructure to entities that wouldn't necessarily appear as top of your mind educational organizations are schools within New York who are constantly being hit by cyber attacks with all the intellectual property that's at our academic institutions the board of directors is from around the world and from all different sectors when we first met at the Federal Reserve Bank the district attorney asked this question what is the first project we should get involved with and around the room the heads of Bloomberg City our partners in France London all without a doubt said phishing to which the district attorney said how boring a phishing this is what we should and he didn't take long to be convinced of course that as many of you know phishing is at the root of most of the problems that exist for cyber and so what the global cyber alliance did is create a series of tools free of charge that one can go to the website globalcyberalliance.org and download tools that can assist in terms of prevention of attacks on email which is DMARC Quad 9 another free tool which deals with threats against the internet and we recently released a tool for small businesses when we looked at what was happening in New York we felt that there was an economic opportunity large institutions could invest in cyber security but the heartbeat of New York is small and medium businesses and they're getting priced out of the opportunity to protect themselves and as everybody knows the large corporations have its chain is based in the small business community so the global cyber alliance is an initiative that we believe in capacity building to take all the expertise that we have with around the world and unify it and produce things that are helpful for our citizens around the world I'll spend two more minutes another global but rooted in New York is the threat to our critical infrastructure so about a year ago the district attorney said in case of a cyber attack he, New York City ready and prepared and the answer was not enough and so the district attorney along with partners with the global cyber alliance the police commissioner of NYPD and New York City cyber command created the New York City cyber critical services infrastructure with one primary mission how do we protect New York City and all the threats around the world when we know we're not doing enough to share that information in real time when an attack against con Edison our electrical provider or against wall street isn't being moved to the other sectors within New York so over time we have now brought together about 37 different organizations again focused on New York but an attempt to address an international problem the threats that we see are both local, national and international but JetBlue wasn't communicating which ASPSC or IBM or Booz Allen and certainly what we have noticed is that the biggest threat is to our hospitals and our healthcare facilities as you all know the United Kingdom went through the crisis of having its doctors and nurses unable to access medical records last May and if you think that if a judge or anybody ever thinks cyber crime is a zeros and ones as Rudy put and we both share the philosophy of course it's not it affects real lives and that's an example of a real life problem so sectors represented in New York small businesses media organizations financial State of Michigan was way ahead of us five years ago started this figuring out how do you prepare at the local level for attacks given that you've got to respond in real time and that if you are going to have a citizenry that is in panic stage unless you have the tools in place when you need to smash the emergency glass at a company to get to your operating plan in case of a cyber attack what's the plan for this city itself New York City 8.6 million visitors 1.8 in Manhattan alone 1.8 million people residing in Manhattan you are trying to stem chaos and make sure that what's behind the glass is a ready made plan for what to do and who are you talking to as part of that have started to do trainings in cyber security to protect our infrastructure and sadly we did have to go to Boston because New York doesn't have some of the things that would allow us to train together meaning state of the art facilities but Boston does an IBM so hopefully New York will catch up finally I'm not going to go into this to too much detail because of time but I do want to say that the initiatives are so promising bringing together institutions people to share and collaborate and give tools but as prosecutors one of our biggest challenges that we face as we try to balance privacy concerns with our judicial orders is the issue of encryption so I'll save that and turn it back to you thank you Ken one of the aspects that struck me from your presentation was for collaboration and the capacity for rapid response not just at the state department and this is just me but I tend to think about it as between and among countries but we have to think federal level with state and local government entities with the private sector among sectors so thank you for fleshing that out and reminding us of that the capacity building has come up in each of the interventions at this point and I think Mary will now elaborate a bit and share some of the experience and perspectives of ABA Rowley in that area thank you Rob and thank you to my panelists and greetings to everyone I wanted to make sure that I bring us back into a conversation that we started during the opening remarks where we're trying to identify what rule of law strategies can be most effective in virtual spaces and I have lots of colleagues in the room from DOJ Optad and ABA Rowley and other implementers and donor agencies so I'm excited to have further conversation after I talk a little bit more about the how and the why in terms of technical assistance and what rule of law strategies we've already designed and developed and what we can do better but you know I wanted to start with the why and I'm not sure we always do that as effectively but the why is as my colleagues have mentioned it's really about peace and security at all levels it's about the peace and security of countries financial physical security of individuals businesses communities hospitals schools and you know the how I have some general observations and then I want to talk about some specific examples I kind of have three buckets of observations some of which have already been brought up by Rob and Ken and Rudy but you know first of all and I lifted from Ken's PowerPoint the motto for the global cyber alliance our philosophy is simple do something and then measure it I think the most effective technical assistance in the rule of law realm are programs that are designed after thorough assessments and assessments that identify gaps in laws resources frameworks and that those assessments are ongoing you know there's logical gaps between the time when we at Roley might submit a proposal in one of these areas and when there's a decision on the funding and when the implementation actually starts so the assessing has to happen all along the way and the measuring you know during the course of the implementation of a program but most critically identifying what you're trying to change and did you actually accomplish that are people safer and more secure are defense lawyers incorporating into their practice are prosecutors incorporating into their day-to-day strategies around enforcement because of what information we shared and the expertise that we helped create so do something and measure it and then the second general bucket of observation is I just kind of call it breath and depth you know to me breath is logical as rule of law implementers especially as part of the American Bar Association is we're working with justice actors and in some countries we're working with a variety of justice actors and including the defense bar but as my colleagues have already mentioned you can't just deal with this through the ministries of justice and the justice actors we also want to work with the judicial training academies with the supreme judicial councils and other entities that are critical at the juncture of implementing rule of law programs and strategies inspectorates the lawmakers and the regulators and then the economic groups and consortias and as my colleagues already mentioned we have to work regionally the breadth has to be wide and strong and in terms of the depth we have to work as Rob mentioned at the local, the national and global level and we want to work with individuals and business owners and global corporations and civil society actors in order to address this the third general observation I have is we have to be strategic thinkers I'm a recovering prosecutor but at times you did not have the evidence you knew something had happened you knew who did it you didn't have the evidence so what else can you do you might file a different Rudy outline for us the variety of types of criminalized behavior has been modeled for us through the Budapest convention you know it may not be the highest level of felony that you can file or you may have to go after someone's permit or you may have to go after their bar license or something you have to try to do something to not if you're not able to do the the enforcement measure that you think is most appropriate and most deserving you may have to do something else I appreciated our keynote mentioning the diplomatic channels you know we've all worked in countries where we've worked with our colleagues from the embassy whether at INL the PolyEcon officer someone who can help us put a little diplomatic or political pressure we can make the progress and the changes that we desire the other the sanctions the asset forfeiture seizures I also my colleague Katherine is here from DOJ OPTAD you know a lot of us have worked together to try to create specialized units multi-disciplinary multi-agency task force and those those are the most appropriate ways to deal with highly specialized areas like we're finding this virtual spaces and cyber issues to be but you can't forget the conservation officer in Tanzania who's probably going to be the one who's going to identify who's trafficking I'm not going to mention flamingos but maybe rhinos or something or the street cop into Blesi you can't forget to train at all levels and at all levels of expertise that everyone needs to have the basic tools that Ken and his colleagues have been identifying and furthering and lastly you know our keynote mentioned the tightly governed spaces you have to find common ground with the actors and the countries and the governments that don't appear to be budging and as a matter of fact appear to be just tightening the news there has to be common grounds and strategies that you can find in terms of furthering the how I have three specific areas of technical assistance that I'd like to highlight from some of our work and I really am excited that we have other implementers in the room to talk about other areas but Rudy spent time helping us become more familiar with the Budapest Convention but basically tying technical assistance this is not new but it's really critical in this area to international practices and conventions and standards a couple examples of that you know a lot of us are familiar with the United States 2030 agenda for sustainable development the development goals and one is intended to harness information and communications to advance gender equality and women's empowerment by accessing the internet accessing information and increasing public participation my colleague Liz is here we had a program in Malaysia where we convened a workshop with most journalists and human rights defenders on the feminist principles of the internet sharing information about digital security and privacy rights online I've been tipped off that the world the second example I give is many of us have helped further mock trial competitions and the largest is the jessup moot court competition I've been tipped off that this year's fact pattern is going to include killer robots in the fact pattern AI okay alright and then my second area of specific types of technical assistance back to the regional work working with regional bodies on a regional basis and a couple examples of that we have programming in southern Africa where we're working with the african prosecutors association and the south africa development community and we're working on trafficking issues in that region of africa thanks to people like rob and other people at INL we have boy I think since like 1999 we've had a regional anti-corruption program starting in the Baltics Central Asia now headquartered in Thailand but through that program our regional anti-corruption advisor coordinated with the US and Vietnamese governments conference around trafficking in wildlife and illegal logging where we partnered with the Asia-Pacific economic cooperative APEC where we brought together over a hundred participants sharing information about how to most effectively combat wildlife trafficking and illegal logging and then the third program we're just kicking off I got to be in Ethiopia last month we're working with the african union where we're developing strategies to enhance the capacity of the staff of the secretariat of the african union to deal with the global breadth of the challenges in their member states my last bucket is I want to talk again about breadth and depth and I don't know if my colleague Ashley Martin is here but we had a two-year program through the good graces of the department of human rights and labor at the state department where we dealt with internet freedom issues and we furthered those efforts that have been described by my co-panelists where we we tried to work we tried to bring in the public sector to deal with public the private sector to deal with public agencies and working with civil society actors much through which we were able to sub-grant the funds that DRL gave us we worked in 18 countries 11 of which were in Europe and 7 of which were in Asia to build the capacity of lawyers and civil society actors in the areas of freedom of expression association and the flow of information a couple things that we accomplished during that two years through our partners there's an internet freedom network comprised of 10 local partners through that freedom network there were 11 country specific state of affairs reports where there was analysis about the gaps were legislatively in terms of resources and those kind of things and then products were developed including in Armenia a multimedia toolkit informing civil society actors, government representatives and internet users writ large about existing and emerging digital threats we created a lawyers network within the that regional body where the lawyers got better at assisting victims of events that occurred in the cyber realm as well as representing actors who were being prosecuted through some of the overreaches of governments in their efforts to try to restrict the space and lastly some of the countries we worked in were very sensitive countries and in one of the areas we brought together representatives from the regulatory agencies private businesses and civil society where we started a dialogue and a multi agency working group to improve the legal and policy framework around information restrictions so in conclusion why are we doing this we're doing it because it involves the safety and security of our partners in our countries the countries in which we work and on a global stage so invite all of you all to share some of your strategies and stories so thank you well thank you Mary thoughtful comments as always and I know I happen to know that you're not only a recovering prosecutor but a recovering defense lawyer as well and maybe at the end Ken would use a couple minutes of your time when he realizes he has to go back home and say that he admitted that Boston has something that New York City does not have so I have to apologize to the audience these interventions have left no nothing interesting to comment on no questions in your mind we've completely settled the issues we can take some extra time at coffee now I'll kidding aside we're going to open the floor now I don't know what the system is for sharing mics but yep we've got a floating mic so again if you just say your name and your affiliation and we invite comments I would suggest that they be relatively safe if it's a comment or questions for the panel if questions will take three at a time so we're opening it up I'm Peter Borgher I'm on the rolly board delighted to see you Mary tell us a little bit about how these various efforts that you've been talking about or what we can learn from the Budapest convention impact private actors for example to take one that's been in the news Facebook is faced with a huge volume of traffic some of which it would like to filter how does it do that appropriately and to what extent are there rule of law either models or maybe actual statutes that would inform how private actors with a big opportunity to impact ought to function thank you Katie Wong with NTD TV my question is about China because as you have said that China is not signed on this Budapest convention but it's also actually involved in this state sponsored cyber attack and also it controls the online freedom of speech and so on so it also promote this concept of cyber sovereignty so how can you deal with China on different aspects thank you for your comments the internet has evolved much over the years I work with the folks that brought you the internet originally and we've been developing new methods where the internet is no longer end to end in the moving forward that was sort of a first cut at how to do it and this gentleman talked about the Budapest and you have the computer data and how do you preserve that well I've had some conversations recently where the digital entity itself the data how do you incorporate it in a computer program because basically you're talking about zeros and ones as you mentioned and if you actually are protecting the information itself and resolving to that information itself you have a fighting chance about how to preserve and manage and at least try to hold back the tide and have the legal access to perform stated operations on those sequence of bits you actually also have a way to manage the global infrastructure and we set up a foundation in Geneva called the Dana Foundation and it's based on an old DARPA project going back 25 years we're actually it's come forward now and we're using the newer security obviously but it's based on the notion of not just end to end where you resolve to the ports machine but you're actually managing the information units itself and I'd be pleased to provide you some information about that now the Buddha's best convention 2001 is there a possibility of evolving the concepts because you're not just transferring records of preserving data data is gotten to be one of those words that what is it information so the information has to be represented in some form to manage over long periods of time the access to those units of information and that's what we spent an enormous amount of time with years on and we're most pleased to share that information with you thank you okay so our three initial questions one and I'll paraphrase is how these efforts have impacted the private sector and I'll add and how's the private sector contributed to the evolution and impact of the implementation of these efforts working in the context of countries that have very different views about the appropriate nature of the internet and role towards cyber crime and issues of the need for our legal approaches and perhaps our conventions to evolve in light of evolving technological approaches and recognition of new needs and issues of preservation of data so I'll turn it over to the panelists whoever wants to jump in first I can jump in just with some remarks on from our vantage point in terms of private sector the initiatives that we laid out and the way that we have to approach short term and long term cyber security issues goes through the heart of the private sector while the government sector of course has excellent professionals and a certain amount of bandwidth in truth the private sector has the opportunity to with 2 billion users at one company have global change in a heartbeat so for us I think the approach has been that we are constantly collaborating with the private sector constantly listening to the private sector and making sure that we are never never missing an opportunity to leverage their good will both at the state, local level and global level we're all in the same same boat trying to address these cyber security issues and so for our collaboration with Facebook we deal with it we deal with them constantly and there are several initiatives that we can talk offline about just wanted to chime in on the private sector issues one of the other rule of law strategies we have identified that important in any rule of law programming but especially in this space are public information and public education campaigns and that's especially acutely needed with the private sector and I'll address the China question how do you deal with countries that are not signatories countries that are not signatories generally fall into three categories you have less developed countries that really don't depend much on e-commerce I don't want to use Paraguay as an example but I'll talk about Paraguay and they usually do not wake up until somebody very important in their country has been the victim of an attack for example in the Dominican Republic that who is a signature they got mobilized when the president a few years ago their president was the victim of a cyber attack all of a sudden what is this and they mobilize now that one of the leaders in all of Latin America the second category would be those regimes are more restrictive the China the Russia for example who are very concerned about maintaining their sovereignty they don't want the Buddha best convention coming in and allowing judges in France to issue orders for China to be obligated to turn over records that's a huge issue and there you have two diametrically opposed legal systems that may never come into an agreement that's just the reality and the third category would be what I would call countries that have a type of complex countries and again I don't want to be specific but since we're talking about Brazil Brazil Brazil is a country that is promoting the negotiation of a totally new treaty and they always come up with a different excuse why they don't want to sign the Buddha best at one point it's because they were opposed to one of the articles having to do with intellectual property enforcement then they came up with another excuse and within those countries there are divisions it's like countries saying hey we're not a third world country we should have been involved in the negotiation of this treaty but since we weren't let's start a new one where we can get in at the ground level but the problem is as I mentioned earlier the Budapest convention took over a decade to negotiate what do we do in the next 15 years while we negotiate another treaty do we just totally abandon Budapest and was join Budapest Brazil and then negotiate any new terms that you feel are important or that may have been ignored but to suggest that we're going to totally abandon the Budapest convention and then negotiate the terms of a new treaty that's not realistic I'll note that among the Budapest conventions 63 or so states parties I may have the number off are both Council of Europe members non-Council of Europe members that are developed countries and a number of G77 are developing countries and then it's estimated that another 60 countries that are not parties to the Budapest convention have modeled their laws on its provisions let's open it back up for more questions and comments and again please just note your name and if you have an institutional affiliation please ask the mics around good morning my name is Jeanette Manning and I'm an attorney with the National Association of Attorneys General here in DC and I head up our international center and my question actually is probably more geared towards you can with respect to you had a slide regarding encryption can you talk a little bit more about advances you're seeing particularly as it relates for law enforcement are they starting to make more headway with private entities now starting to hopefully cooperate as it relates to investigation so can you elaborate on that good morning my name is Orlando Munoz I work with ABA Rolly in Colombia South America and my question is to Rodolfo could you please provide more details about the Argentinian case of the I mean how Argentinian authorities acted against the way of the game good morning thank you again for giving us this panel my question is for Mary because you mentioned the feminist principles concept of internet security I was just wondering what are you seeing as the major challenges or gaps in the ability for law enforcement to bring cases specifically around cyber mob attacks those that limit women's participation in the internet when those sorts of online mob gathering in an organization lead to physical violence or threats of violence thank you so we have encryption Argentina I was terrified for a moment that you were from Paraguay and law enforcement responses to cyber mobs and online aggression against women alright I'll start with Argentina one of these cases the Argentinian authorities got through social media specifically through Facebook this message and they recognized what it was they try to reach Facebook to get subscriber information under the emergency exception there's an exception under our law that says when you have a life-threatening situation where the potential is imminent that someone's going to lose their life service providers may in their discretion provide records that are necessary Facebook refused to turn them over and this goes to one of the questions that were asked earlier the role of private the private sector so they contacted the Department of Justice that is Argentina did to help to see what we could do we recognize that this was an emergency so at the time I was working for the computer crime section I called Facebook and they said this is not an emergency my response was you know what this is your decision but if somebody ends up dying and they ask the Department of Justice why we didn't act and the impact that may have on the value of your shares whether they go up or down that's your decision we got the records within two hours and with those records we were able to find the geolocator information the geolongitude information and I was able to actually see the house from which that Facebook message had been sent which I provided to the authorities they identified the individual he was a 16 year old very disturbed individual they found out the other people he was communicating with through his computer and as I said they were able to save about six lives thank you for the question on the feminist the law enforcement issues around cyber attacks on the internet threats of violence I don't have any I turn to my colleagues here and in the audience for any specific examples from a rule law but I'm a strategy or issues but I'm imagining the investigative challenges are real and it's what we've already identified the access to electronic evidence in real time let me start with political will by either governments or investigative agencies the challenges with obtaining electronic evidence threats of violence are definitely harder from an enforcement standpoint than actual assaults or actual stalking, actual hate crimes and then the attribution issue which my colleagues that are better the use of electronic evidence can maybe maybe elaborate on so thank you very much for your question it actually will allow me to tie back to your earlier question with the audience permission I'll do a two minute primer on encryption my comments throughout are never criticizing any particular entity it is an observation for you in terms of what the impact has been for us in the state and local community on the very practical path that we have which is to investigate, exonerate and to prosecute so this issue for us the going dark issue has been around for decades but it hit the state and local community most in a few years ago in 2014 because from a data at rest perspective your phones, these are the devices that we typically in the past have been able to in every instance go to a judge get a warrant, take that warrant take that phone that's associated with a private sector ship it off to a California entity and based on the judge's authorization get into that phone as we're focused on human trafficking, sex crimes whatever the case a judge has read our application and made a decision but in 2014 Apple made a decision to change its operating system that's their private sector choice that was dramatic to say the least now why does it matter is because Apple and Google control because of their market share about 99.3% of the operating systems around the world for our office as I talked about in my initial comments we run a state-of-the-art lab very proud of it no matter how many terrific professionals we have or the tools we use we can't get into most of these devices more than a thousand devices that we lawfully obtained since 2014 were inaccessible when they were seized more and more of these devices because of the operating systems just come in and they are inaccessible to us after we get a warrant again remember we have the legal authority but not the physical ability to secure it where are these cases coming from these are 9% of them are homicides attempted homicides real life issues 10% of them are sex abuse cases including child sex abuse human trafficking around 7% and then the unique thing is from a cyber crime perspective these cases about 30% of them are related to cyber crimes so we can't get into the phone trying to tackle the evidence to just examine it we could exonerate but we can't get into the cyber crime so we are asked to investigate but we can't get into the evidence let me just talk a little bit about what the practical implications are videos on these devices affect our investigations of sex crimes human trafficking when we look at these these videos were strong corroboration of the victims of 13 year old girls narrative the defendant entering her bedroom and assaulting her talk about the most dramatic cases when I talk about exoneration and the need to get into these devices I'm not playing lightly I'm saying we've looked at this and in 17 instances because we were able to get into the devices a man or a woman who was accused either had their charges completely erased or they were reduced so to your core question are things getting better or worse? worse much worse can't get into the devices unless we spend a lot of money to outside consultants to do it and we when we're dealing with not data at rest but data in motion when we're trying to do our wiretaps we're frozen out so this is not a tech problem no one's pointing the finger at the tech community this is not a law enforcement problem this is a societal issue so we're all in this trying to grapple with how do we balance public safety, threats to critical infrastructure, threats to all the things that I talked about on the earlier initiatives and privacy and we are stuck we're in a really tough place where the victims of our cases we cannot get to the evidence and so what I will say is we have put together every year a white paper on the subject you can just visit our website ManhattanDA.org to access those we approach it as neutrally as we possibly can but with a clear underscored emphasis that from a public safety position this is untenable so to serve back to your question about the private sector in every sector except for the tech sector we have a robust at times there can be friction but in the tech community it has been really difficult now that has gotten slightly better slightly better over time but not much so I wish I had better news and sorry if we take up so much time so we started a little bit late but I don't want to take us too much later or go into the next session so I'm going to take one minute and just mention some of the themes that jump out from the presentations and from the discussion of the questions and then we'll recognize our panelists so from my perspective some of the themes that stand out are the need for adequate legal frameworks as crime has evolved the idea that having common elements among countries helps with international cooperation and the importance of capacity building and thoughtfulness and drawing upon lessons learned in capacity building the importance of not just thinking as I unfortunately often do of federal entities or country to country cooperation but sub-federal entities or sub-national entities and cooperation that is vertical and horizontal and the innovative approaches that states and localities have in confronting these issues the fact that we have to treat it as a multi-stakeholder issue involving the private sector and thank you for raising the point of the constraints and the challenges for small and medium enterprises that we don't always think about we often think about the behemoths in the field in all of those directions that we mentioned and a planning and preparation the point that a criminal justice response in a sense is a response when it's too late and we're in a defensive posture and then, and this could be another hour or two hours of discussion the balance of privacy and security and the challenge of encryption so I'm sure that you are with me as a panelist to be articulate thoughtful comprehensive responsive if you'll join me to both thank them and thank ABA and USIP and I'm sure they would be happy if you button hold them for more side discussions why don't we get started now my name is Latha Knott and I am the Executive Director of the First Amendment Center which is located right here in DC at the museum and is an organization dedicated to exploring issues related to free expression which is why I'm very honored to host this panel today which is the tipping point when is cyber incitement responsible for violence and our goal today is to have an informal discussion about the dividing line between free speech and hate speech and what factors can lead hate speech to result in actual real-world violence I think that I won't get much argument when I say that there are a number of paradoxes that surround the conversations that we have online they are hosted and moderated by private platforms but they have a huge impact on public discourse they connect people all over the world but you can see that they often lead to a polarizing effect in so many societies and while social media can be an invaluable tool for spreading news and information after a crisis it can often be the cause of crises when you see it being used to spread misinformation and incitements to violence as a free speech advocate I'm generally in favor of more speech rather than less speech and I'm wary of censorship but I would deny that social media can have an amplifying effect on violence in its users that it is susceptible to manipulation and that it can have a real-world cost a human cost when unfettered communication does lead to violence and it's always been very difficult to balance free expression with security and safety and luckily for us we have two experts today who can help us parse some of the things that we do when doing so and our first expert is Richard Wilson who is the Gladstein Distinguished Chair of Human Rights and Professor of Law and Anthropology at the University of Connecticut and we also have Stephanie Klein-Albrandt who is finance and economics the finance and economics expert on the UN panel of experts established pursuant to UN Resolution 1874 and the plan for today is that both Richard and Stephanie are going to present their initial commentary on the research that they've done in this area they'll talk about the case studies of Myanmar and Guatemala Richard will walk us through the framework that he has developed to assess when hate speech is likely to turn into violence and Stephanie will talk about where we can go from here with what we know about online speech and violence what are our next steps what course should we plot after that I'm just going to ask a couple of questions to get the discussion started but I'd like to turn it over to the audience for your questions following that but first Richard speech against human rights defenders in Guatemala with the ABA and behind me is a team in this report my colleague Molly Land who unfortunately is ill and couldn't be here today but she has been more than an equal partner and also the research and much of the writing for the report has been done by students at the University of Connecticut School of Law two of them are here I can't help but embarrass them because that's what teachers do ask them to stand up Mona Abbas and Jonathan Donovan who just graduated if any of you are in positions of authority hire her that's my recommendation and also the support of the ABA Center for Human Rights team who have just been amazing in ushering this along Brittany Benowitz, Jenna Anderson and Juan Ramirez have been just amazing to work with so we are talking about hate speech online and it is a tidal wave of a problem but it is a huge problem if you see just to point out some of the some of the numbers here 4.75 billion pieces of content shared daily on Facebook just all these hours of video uploaded on YouTube Twitter 500 million tweets a day and 80% of users outside the United States so it is important to recognize that the social media companies and their platforms are not just U.S. phenomenon in fact they are not just predominantly U.S. phenomenon this is a global phenomenon and that has all kinds of impacts for the regulation and law that applies here are some more numbers for the first quarter of 2018 I will just point out a few interesting facts here 583 million fake accounts I believe that number has gone down but that is an awful lot and 2.5 million pieces of content featuring hate speech which is our topic for today Facebook has community standards in their terms of service they have three tiers of content moderation the most serious tier is tier one there are a number of protected categories like race, ethnicity sexual orientation gender, disability and these are protected groups and calls for violence disease or death dehumanizing languages in tier one mocking hate crimes endorsing them tier two would include making claims about the deficiencies or mental physical deficiencies of these groups holding them in contempt calls for segregation and this also includes ethnic slurs or other forms of language that includes slurs so there is a kind of hierarchy here that the social media companies have indicated to us of what they consider to be the most serious forms of hate speech and what forms of hate speech they consider to be less serious so we can argue about this how did these categories come to a rise are they the best ones to have you know this is certainly a topic that we can discuss there has been criticism that these hate speech guidelines are over broad but also under inclusive over broad in that napalm girl a photograph which arguably ended the vietnam war a photograph from 1972 in vietnam was taken down on facebook in 2016 it was later allowed but it was taken down under the hate speech guidelines and then other kinds of posts which perhaps deal with immigration are allowed to stay up so there is an argument here that the hate speech guidelines are not getting more coded forms of insult more coded forms of speech but they are actually excluding forms of speech that we ought to allow as part of our democratic and public discourse so we focused on hate speech in just one country in our report which is Guatemala why Guatemala well it seemed to be one of the more egregious cases but also a case that we wanted to dig into more deeply but there was simply a problem here of a number of problems I'll just put them up here which are present also in other countries Serbia, Colombia, Cambodia come to mind this is not just a Guatemala issue but one of the problems was that prominent state or state aligned actors were targeting human rights defenders there seems to be a problem here in social media companies approach to hate speech which is to treat everyone the same everyone is just an individual out there making their posts but what happens when there is a coordinated state or state aligned program of targeting a particular sector of society in Guatemala the language was labeling human rights defenders as communists or terrorists and also explicitly to encouraging violence from the security forces this harmful speech was by and large not being removed by social media companies and there is a high level of violence against human rights defenders in Guatemala in 2017 20 human rights defenders were killed in Guatemala is there a correlation then is there a question between the targeted harassment intimidation and incitement by state and state aligned actors and the actual violence on the street against human rights defenders it seemed like there was a correlation in the violence in Guatemala and 18 when president Morales called human rights defenders criminals and 8 were killed in the subsequent months so we dug into this problem we researched it for 9 months now and here are some of our findings first of all there seems to be a serious problem in Guatemala and also in other places of indirect speech which doesn't explicitly violate the hate speech guidelines or terms of service of social media companies but nonetheless can have deleterious effects it's coded it's veiled it's evading content moderation now since we're all users of language we know how complex and nuanced language is we know how language within particular community groups can be used in ways to secure the message from other outsiders to transmit a message that only a certain group will hear in the United States we call it dog whistle politics but it's happening everywhere and always has in political discourse but this seems to be a particular problem for social media companies how do you pick up on indirect speech where the audience knows exactly what's being said but it's not being said in a way that it is grabbed by the filter of the hate speech regulation of a social media company the messages were then amplified in Guatemala by net centers and fake accounts we found extensive evidence of this and there have been a number of reports including one yesterday by the UN mission in Guatemala on net centers and you can download that at ccig which is the UN mission against impunity in Guatemala and it just produced a report on that topic it's not clear from our research this doesn't mean it's not happening but it's not clear from our research that there's a direct correlation that would satisfy a court of law that a specific speech act cause a specific criminal act that's what criminal law looks for it looks for that causal nexus and evidence beyond a reasonable doubt it's unlikely that we're going to get that kind of evidence if we look at what's happening on social media online if that's our standard we're going to be frustrated but we had reported indirect effects from human rights defenders in Guatemala intimidation, silencing harassment, stigmatizing being fired from their jobs and not able to get other jobs long term consequences which created conditions in which violence towards defenders is tolerated one human rights defender when we said what's the kind of impact of this delegitimizing and harassing of human rights defenders and their response was to say well you know if something bad happens to us then maybe other Guatemalans who don't know about the situation say ah tiene que ver algo they must have done something there must be some reason there and that's the kind of important effect of political discourse especially of the indirect kind which is about creating a tolerance for violence creating an elevated willingness on the part of the listener to engage in moral justifications of violence that's not directly causal to the violence it comes in a sense after the fact but still can have consequences and something else that we find is that human rights defenders are not a protected category in any of the social media platform filters so here are recommendations we think there ought to be not a single algorithm for hate speech that's what the situation is now that's the status quo social media companies want to create a list of words that they can then regulate and filter across the world evenly we would argue that that model it does respond to the question of scale but it doesn't respond to the question of the contextual needs of a country that's going through armed conflict a constitutional crisis possibly genocide and there we would argue that what we need are a kind of bespoke policy where a social media company designates a country, a critical country where there is no rule of law where there's massive levels of social conflict perhaps a constitutional crisis as there is in Guatemala right now and that these are designated critical countries and in these critical countries you get a team put together which engages in context specific content moderation what does that mean it means that the speech is evaluated by native speakers who can see the indirect language the coded language that's being used the speaker from Facebook earlier talked about adding 30,000 individuals who are engaged in cyber security and content moderation that's great but what we find is that individuals are sitting in the Philippines or in Dublin or in the Ukraine and they're moderating content all over the world and sometimes they're using Google Translate so even if you're a Spanish speaker from Spain you may not pick up on the slang of Mexico City or the slang of San Salvador or what's being said in a rural community in Guatemala we all know how variable English is for a New Zealand speaker earlier we all know that there are these interesting little complexities and nuances to language which only a native speaker is going to pick up and we recommend that in those in these kinds of situations where there is a crisis in a country that those native speakers be brought in that the speech is evaluated alongside social and political contextual factors so you can't just look at the speech and this is consistent with 100 years of First Amendment law in the United States since 1919 and all of our Wendell Holmes in Schenck the US Supreme Court has been saying you don't just look at the speech you look at the context of the speech and so all we're emphasizing here is that social media companies need to contextualize the language that they're regulating and filtering what are the social and political factors in a country perhaps look at those in order to understand what the consequences of words can be and in situations which are of extreme crisis it's certainly the case that there are determining factors which are risk factors which elevate the likelihood that violence is going to occur elections are one a common one there's much more inter-ethnic violence inter-political violence around elections the authority of the speaker is another there are a number of factors which are risk factors which are indicative and predictive of the likelihood thirdly look at the status of the speaker are they state or are they state aligned we request and recommend that content moderation be sensitive to coded or veiled speech and that there be de-platforming of accounts that violate the terms of service this is what they're meant to do anyway and this might be done in a more consistent fashion as a result of our investigations and report one Twitter account the foundation contra el terrorismo which is one of the worst accounts in Guatemala has been suspended and closed but which we may consider a victory but then there's the question of what appeals process is there we'd like to see social media companies being very explicit about the reasons for de-platforming the reasons for removing content could they explain that could they give us clear criteria when an individual and provides some kind of appeals process for them thanks very much thank you so much I'm revising my speech a little bit to take account of some of the points that we had in common so I'm going to speak today both about platform governments and just a little bit of time on government repression and use of the internet by authoritarian governments to suppress rights the main rights at stake are right to privacy and freedom of expression and association and there are others as well but those are the main ones that I'll address with the privacy rights the main impediment is really just connected to the inherent online business model which relies on extraction of information to better target ads and freedom of expression the boundaries are defined in a kind of a gray zone between legal frameworks and company norms I just want to start with a quote E.O. Wilson said the main problem with humanity is that we have paleolithic emotions medieval institutions and god-like technology and this has led to really an extractive attention economy with reverse engineering of human instincts that's sort of where we are today and what this requires and I'll get into some recommendations in a bit we really need action by the companies by governments and civil society to better protect speech in the digital age and not merely incremental changes at the margin of sort of social media's management of public space we also need changes to the rules governing online speech and public participation in making interpreting and enforcing those so just quickly with regard to right to privacy so these the relevant issues include expanding surveillance regimes tracking and profiling of users online behavior data transfer without required safeguards if you're all familiar users can increasingly now make tweaks to how they share the information there's really still no means of limiting the data collection that is the premise for the services that are used so targeted advertising what's really scary here is that we're really even way beyond Cambridge Analytica at this point because you don't even need any technical tactics to access a huge amount of information about users right now it's possible to assess the top five personality traits of an individual based entirely on their click patterns and their mouse movements and it's likely that that information will be given to an insurance company before you yourself like they can find signs of Parkinson's years before you have any actual real symptoms so it's likely to be given to a sole-to-insurance company to a government before you as an individual will be given that information moving on to freedom of expression so the platforms like to speak as though they are enablers of freedom of expression and association and you've heard how that's done connecting all these people giving rights giving free speech to a whole host of people but they effectively have both boundaries for expression based on mostly company created norms or community type standards and legal standards but we don't really know the actual mix of all of that and then they regulate content through different forms of content moderation so there's algorithms there's user flagged content there's requests from governments and other state actors and agencies for removal of content now the algorithms have been successful to a large extent with things like child pornography, terrorism violence including suicide but not to repeat too much what Richard has said but it's really much less successful for hate speech for the obvious reasons of the use of allegory the use of nuance the use of coded language for example before it took the outside world the expression Rohingyas are Bengalis was not something that Facebook was looking at it's literally kind of dehumanization of the Rohingya to essentially say that they are not Burmese and that's just not picked up and if it's picked up even if it's a day or two later the damage has been done because these viral posts already calling for violence have been disseminated there's another case of I think it's South Sudan or Sudan where the persecuted group is referred to by the acronym, three letter acronym referring to the telephone company because quote they're everywhere which is in and of itself hate speech but how long does it take for a platform to figure that out and then you have to actually understand the process of creating an algorithm first you have to have human beings agree to what hate speech is which is totally in and of itself a difficult enterprise because defining what hate speeches can be left up to all sorts of different types of interpretations then you need to create rules about what hate speech should be allowed so Richard pointed out the tier one tier two and tier three then you need to build the rules into algorithms and then you make the algorithms observe the rule and then you need humans to constantly review the algorithms to make sure that they're actually picking up the content that needs to be flagged so it's a long process and it locks in definitions from the very beginning and requires constant review and yet as you might all know I mean the types of information that's fed in to AI is sometimes very questionable discriminatory faulty and there's kind of an assumption that AI is this magical process that cleanses it and makes it super technologically advanced so what comes out must be somehow correct and we've seen this there's been studies in the use of AI by police stations around and it shows that it just furthers discrimination so with regard to hate speech in August 2018 so the platforms have decided at least one certain platform that algorithms are really the primary way of dealing with the situation hate speech in Myanmar according to Facebook the amount that they were able to remove went up to 52% from 13% and the goal is just you know let's get up to 90% but it's almost a physical impossibility because if something starts as a derogatory term in a suburb of Lagos tomorrow how long is it going to take Facebook to pick up on that and how many posts will there have already been and there already have been deaths and what sort of tragic is that police in many of these countries just follow the platforms because they know that the violence will follow what's on the platforms and do they have the capability to actually have you know their whole police force monitoring the platforms and doing policing so what we need are changes to the rules governing online speech in the public's participation in making and enforcing those we need actions by companies, governments to protect speech we need new models of content moderation and public oversight and government regulation all of which would protect the rule of law with a long term vision of public investment to sustain the infrastructure of freedom of expression I'll just get into a few ways so decentralized decision making companies are addressing problems of scale by hiring or promising to hire more moderators with language skills 30,000, 40,000 right but there's really extremely, extremely so that's on the sort of on the language side these people can maybe read if they have it for a given country because generally they only hire start hiring en masse after there's been a scandal in a country but assuming that you can they can access and read it there's limited participation for platform users to help create or enforce standards in the context of the cultures in which they operate every community is different they have global human rights standards and they're implemented differently according to national contexts so there needs to be better interaction within coverage from the field you need more people on the ground that know what's going on putting them all at the level where people are actually impacted by their decisions and have some role in governing them it's like a governance question so in addition to sort of content moderators and oversight board one can imagine actually having employees who have knowledge in the country concerned and not just helicoptering in from California or from wherever this idea of kind of California overlords making decisions like the decision to pull the four ceasefire groups off the web in Myanmar who was making that decision on basis of what did they actually find out what this means why those groups and not the other groups why these ones are particularly targeted by government implications for elections how much contextual knowledge was there in that decision apart from an antiseptic treatment of probably a package that was given by the government of how these groups are engaging in violence with none of the context around it this would also help platforms to figure out how much leverage they have in a given market because the requests for taking stuff down from governments needs to be evaluated in context of how much leverage the platforms have if it's widely adopted by people you can be reasonably assured that if you refuse certain requests that you're not going to be shut down for example but you need to have the local contextual knowledge to do that to make any of those decisions human rights standards should ideally be the explicit standard for content moderation guidelines now this is kind of a no-brainer right the community standards incorporate some aspects of human rights law but wouldn't it be stronger if you went to a government saying we are making a decision to pull this content off or to take this action in line with international human rights law to which you are obligated and we are as well by virtue of principles on business and human rights GNI etc then these are our guidelines that the whole community agrees with and this is why it should just be made explicit and I understand that that's a heavy reach although friends of mine that work in these companies in the policy realm there's a lot of people that are very much on board to do this at the top level there's no agreement that there should be an explicit simply even if you can't make human rights standards the basis maybe just a reference in the community standards or in the standards themselves that this is based on customary international law as articulated in the following human rights instruments so it seems like a no-brainer then there's a problem of transparency lack of transparency right we don't actually have the data on all the harm that's being created notwithstanding some great platforms that tell us every day the research that Richard and others are doing or global voices it's not like there's an oil spill right and and you have to you get to see the actual data now you have companies publishing reports right you're getting more and more information on what they're asked to take down decisions made etc but they still it still requires more more steps and I would argue that those are to disclose policy choices and applicable content standards right to open up the processes and proposals to public comment and when new rules about content are adopted explain how those changes were made decisional transparency so the reason behind content actions basis of a decision and how a user can appeal again this me and Marcia's fire groups decision what was the basis why these four groups why these other four groups that do the same thing were not targeted what was the basis of that decision provide some clarity into algorithmic decision making so the inputs into the AI like I said input if you input bad you're getting output bad actually if you input bad you're getting output worse right because it's just amplifying the bad data that you're that you're entering and then ideally companies would establish case law right which would open up as much as their decision making as possible to public scrutiny and you'll all have heard of the oversight board that Facebook is setting up kind of like a Supreme Court for Facebook and it's a positive step it's a very positive step ideally that would be a platform like an all platform wide body that would be truly independent because it will be paid for by Facebook anyway even if it goes through a third party it's clear that those people are still it's not truly independent but it's a good step I think I had a lot more but I think I should maybe stop in terms of time yeah or maybe I'll just very quickly very quickly talk about government regulation so the government regulation that's needed is regulation to monitor company behavior protect the space for individual expression reinforce this need for transparency by the companies and themselves and investing in the infrastructure necessary for freedom of expression in their countries and you know there's there's four leaning steps that should be part of any regulation of social media which would require company disclosures companies can edge towards better transparency in part because of their fear of government regulation right but even if they start to do so governments have good reason to demand that the companies disclose their rules and decisions which should give their public the tools to decide whether to engage on the platform and Germany's net DG has begun to do this companies have been producing specific reporting about net DG's implementation this is a kind of regulatory move that could encourage greater public understanding of platform operation and we might get in the discussion the extent to which Europe is really ahead on a lot of this and that we should be learning lessons from what they're doing the other thing that's important is that when requests are made to remove content from the platform it would be important that the platforms require that that comes from the platform. If not ideally a court of law and not just some random police station or individual or agency that doesn't like a certain thing there should be a process whereby our institutions are involved where there's a judicial review of the decision before that's removed and ideally again that that be transparent that process right which governments are asking platforms to remove which kind of content and finally something I'm very passionate about is when the content is removed where is it being saved for future war crimes trials and I know from my work in the UN that this question was posed to Facebook early on in the Myanmar saga and the initial the initial reply was well we never thought of that I think that now they're thinking about it but just imagine what kind of a job it is to be able to maintain and categorize that information it's valuable information about things that are happening that are subject to international jurisdiction for example so that's just another issue that we have to get our heads around and with that I'll stop Thank you both so much for that a lot of troubling knowledge I think for everybody here but that was very interesting and I just wanted to ask you guys a couple of questions before I turn it over to the audience first interestingly it sounds like both of you are talking about the value that there might be to online speech if Facebook were to have more of a legal process if it were to say have an appeals process for people who would be platformed and have its own body of case law do you think that's where they are headed? I don't know the answer to that possibly it seems like there's a hodgepodge of hate speech guidelines my understanding is is that there are 200 pages of rules and documents that content moderators are using to consult when they review a post that it's not consolidated that it's not consistent if you look at the tiers, tiers 1, 2 and 3 that those would not be the forms of speech that I would put into tiers 1, 2 and 3 that if you look at the social science of hate speech and incitement and there's a lot of new neuroscience on this I would look at Susan Fisk and LaSonna Harris's work for instance Susan Fisk is at Princeton and doing amazing work on this stuff you know we might come up with a different a different set of categories I think that the companies are looking to governments to provide some of these answers and finding it very difficult to come up with it themselves and where governments have come up with those answers and we can talk about this the German Nets DG law for instance the response of the social media companies has usually been whaling and gnashing of teeth but then compliance and then a sense that actually there hasn't been a major violation of democratic rights to deliberation and due process and discussion it's no one understands this completely it's the Wild West and I would like to add to that the place where we're seeing more legalese is in this oversight board discussions where in theory you could have case law built up and you could then by the case law if it were transparent extrapolate back exactly how the platforms are reasoning Article 19 is called for social media councils and again this would be something that would be across social media that would have a model that would work for all that would be obviously something more consistent with a legal approach than having each company have their own properly funded their own like funded by themselves institution that in theory is providing oversight and I think we're also talking about when we think about Facebook having a legal system it going a little bit going further than say the first amendment I mean do you think that there is just an inherent difference in online speech versus speech in real life definitely all this stuff violates the first amendment all these hate speech guidelines because the first amendment indicates you can't have content discrimination and so this is clearly picking out some forms of content you know there the social media companies are essentially clubs and they can do what they want with regard to their terms of service you accept them or you don't at the same time they're not just clubs they are the internet in places like India and Burma and other places and they're much closer to public utilities so I think there's this real tension between their private nature and their public function and that's not something that's easily resolved yeah I'll just point out a different issue which is the difference between protecting speech that's not online speech that is online so your drunk uncle in the back of the room making ridiculous racist comments is one thing and putting that online when you know it's inciting hundreds of thousands of people to commit violence against those people so you just have this amplification aspect where crazy ideas are given a megaphone in a way that creates off platform situations of violence and other abuses of human rights I mean there are other ways of responding as well we've talked a lot about deplatforming and removal of posts but there's all kinds of interesting nudges and here you might get into behavioral economics which is short of censorship but if someone uses an ethnic slur in their twitter post a little a little box could pop up saying this violates our terms of service are you aware of that there's also a group which I believe is based in Europe, Sweden and Germany and they intervene in debates and call people out when they're using highly incendiary language and saying really is this helpful and it's interesting how much that kind of shaming and norm enforcing can be effective but the group argues that when they do that 80% of the posts are actually taken down by the users that posted them they're shamed so it's dialogue, shaming dialogue rather than censorship I see what you mean we've got about 10 minutes left so I would like to turn it over to the audience to see if there are any questions and I've been told that we have at least a couple of microphones running around is that true all right can we get one over there I had a question in regards to maybe some futility in trying to take down hate speech and I think you kind of spoke to that earlier both of you in terms of the shaming idea and that aspect in the sense that when we take down hate speech that kind of pushes it towards darker corners of the internet that we can't regulate towards platforms or even creating one's own platform on which to kind of create that echo chamber and push those ideas in the community that they're accepted I think that maybe what's even more dangerous than the extremist hate speech are people who are kind of on the margin or susceptible to turning towards it and when hate speech is in the public you have the chance to delegitimize it and speak to it on a public platform in a way that those people can see faults in it or see flaws in it or even see it as a crazy person with a megaphone if that is the case but when we just take it down do we not legitimize it and give it power and if that is the case then how can we make sure that in taking down hate speech we are delegitimizing it instead of pushing it to corners where it can grow and give it power so great point and much of the speech that we find offensive must be dealt with by counterspeech absolutely and that's a core principle of the First Amendment for the last hundred years and I agree with that and Nadine Strossen has written a book called Hate which came out last year which she makes that argument very successfully I think at the same time I think there's a difference between a private individual who's your drunk uncle, proverbial drunk uncle who's contankerous and then state aligned or state actors in Burma it was the generals calling for the genocide against a Muslim population as part of a long standing set of tensions between the Buddhist and Muslim population in that area that I think is different and maybe handled differently than the drunk uncle who's simply sounding off but you're absolutely right that 4chan and 8chan are radicalizing environments the shooter in Christchurch the shooter in the Powe synagogue are citing other actors on 4chan and the posting of the actual video of the shooting happens there the manifesto it gets circulated there so there is clearly a problem with suppression and mainstream environments sending it into the dark corners of the web where it then thrives just a few words about the internet as it's grown and evolved is the dimensionalities have taken on a scope that people say oh we can't regulate it or if you have for example going back I'm familiar going back as an attorney to the days when you were innocent when you had broadcast programs and there's actually a convention called the Brussels satellite convention where was the information mapped into signals communicated via direct broadcast satellite I was working at the UN in Paris at the time I was part of my job it came up again recently in the trade agreements the US, Canada, Mexico so in that context when we went to the diplomatic conference they didn't realize it was a broadcast it wasn't war mongering or signal jamming and they wanted to turn it into something to control the actual information itself I don't call it content it's the information represented in some form and at the time they went ahead with the Brussels satellite convention and they referred to the outer space committee of the UN to deal with the more political issues about war mongering and jamming it's much the dimensionality is larger because a lot of people can communicate more than they ever could before now when you look at it today and you see what they call platforms they're information providers and as an information provider they could be good or they could be bad they could be state actors so I would be very reluctant to actually have as a legal framework run out of private groups that may be actually state actors so there has to be the rule of law and that's why we're here today to talk about you could pin it off of say the declaration of human rights that's grand but how do you enforce that how do you take measures to prevent those information sources bringing information in how do you stop it at the border oops there goes some bits what are they but there are ways to do the management of the information itself right now that may improve this but I just wanted to say let's not say it's an impossible to do this now there's an example recently where they actually did it I'm a copyright lawyer so they had the notice and takedown in the copyright structure based on the white boat treaties to indicate your information is not really openly public so if you did you can actually ask the platform to have a notice and takedown but then there's a legal process if something is not carried out properly if you didn't if you said to take it down you weren't authorized or if you were the platform and you somehow wouldn't obey it so what I'm trying to say is that there are examples and to think it through before simply saying we're going to turn it over to somebody that's not under a legal framework of some sort that was more an observation than it was a question sorry that's right thank you I think we have time for a couple more questions can we get a microphone over here I think I might have grabbed it first that's okay my name is Adeline Hyde I work for the Washington office on Latin America and this report actually on the case of Guatemala it's very timely and I'm just wondering I have a quick question while we wait on some of these models to be adapted or accepted on a larger scale how in a worsening political context like that of Guatemala what can we tell individual human rights vendors or organizations that are experiencing this and might in the next few months have a heightened experience of this in this moment as someone who doesn't work on these issues as frequently thank you so did you hear the question she works in Latin America what can we tell human rights defenders in Latin America or elsewhere about how to respond to these issues that was a question we asked social media companies and Twitter said that they were engaging in a number of training exercises it is clear that training can help in greater literacy on social media to know how to flag to know how to respond to a program of harassment but it's also very difficult to say to someone if you're experiencing a troll storm and you've got 10,000 bots hitting your Twitter account is there anything you can do about that probably hide under a rock for a while but it's clear that that then triggers a responsibility on the part of the state to if it is neglectful of a pattern of abuse of intimidation and harassment then that does trigger international and national obligations legal obligations on the part of a state to protect its citizenry and we don't allow no country allows incitement of murder you can't do that and if they don't enforce it or they enforce it in a way that is only for some people and not for others like human rights defenders then that's a problem and so then one can argue that it could be taken to a higher order I'd love to see the Inter-American Commission on Human Rights take this case I'd love to see the Commission or court probably the Center for Human Rights thinking about this I thought about this months earlier so I'm just behind the curve here but yeah that would be really interesting to get legal guidance on those kinds of questions and one last question a sort of a practical question earlier you mentioned that how in the US or in certain countries Facebook and other social media platforms are essentially clubs and are private they can make their own terms while in other places they're more public and I wonder if in other areas of our legal system sort of classifying social media platforms as public accommodations for example like for purposes of the American Disabilities Act classifying Facebook as public so they have to comply with web accessibility if there is a potential impact in doing so in that realm and how that could have an effect on whether or not Facebook is allowed to basically have these terms that might be seen to violate the first amendment it's a great question so the question is to what degree would designating companies as public companies or public utilities allow greater enforcement for instance of the American Disabilities Act it's a great question and I could see that what you're describing could be a useful process to explore but here's my my reticence and that is if for instance these social media companies were designated public institutions or public utilities in some way then they would be like public universities and the ACLU has a 100% record of suing public universities for setting up hate speech guidelines Michigan being perhaps the most prominent example when University of Michigan created hate speech guidelines in the 90's they essentially just ended up writing a check to the ACLU for their legal costs they lost that one and so there the very capacious framework that social media companies have created for regulating hate speech would go away and it would become much more constricted it would go right down to the first amendment which allows certain kinds of suppression of hate speech for instance Virginia versus Black 2003 the cross burning statute of Virginia was upheld at the Supreme Court because it only referred to intent to intimidate whereas the Michigan statute intent to intimidate on the basis of race or ethnicity was struck down because that was content discrimination so then we would have a much more restrictive framework if that were the case and do we want that open discussion thank you all so much for coming to this plenary session thank you to our panelists as well everyone I have a housekeeping announcement you've noticed that we have far more to say than we have time to say it I'd like to ask your cooperation in helping us recover some time for our afternoon panels by trying to commit to a 30 minute lunch the afternoon will be just as interesting I promise thank you so much good afternoon so I guess we'll get started my thanks to Linda Beshai and Mary Greer and the entire ABA ROLI team for hosting this great event and inviting me to participate in it I'm Steve Kelly I'm FBI's chief of cyber policy and I'm pleased to moderate this panel on cyber crime enforcement before I introduce our esteemed panelists I'd like to set the scene so a cursory glance at the news on any given day will inevitably yield a major story on the cyber threat indeed many of us have become desensitized to the major data breach announcements stories on election hacks foreign disinformation campaigns compromise of national security secrets and the penetration of critical infrastructure control systems the single factor that makes cyber threats more challenging than all others is the fact that the perpetrator is rarely in the same country as the victim an internet infrastructure and traffic traverses national boundaries in very inconvenient ways in response nations around the world are building up their capacity in network defense law enforcement their intelligence service and their militaries to protect and deter against these threats and increasingly working together this morning we heard about norms of responsible state behavior in cyberspace fighting crime and addressing malicious cyber activity emanating from one's own territory is a fundamental responsibility of states and while solving cyber crimes remains challenging the U.S. and its allies have had success in gathering evidence and making cases with the assistance and support of victims retained cybersecurity firms and outside counsel this panel explore the current state of play in cyber crime the importance of cooperation by victims experiences in international law enforcement assistance the U.S.'s recent experience in charging state actors with computer crimes and the effect these efforts are having to reinforce norms and deter destabilizing cyber activity so let me introduce our panel we have four guests here covering the spectrum of cyber crime prosecutions all the way through private counsel to victims so we have Mick Stawa is here next to me deputy chief for cyber crime with the justice department's computer crime and intellectual property section next to him is Will Lyne he's a cyber crime expert and a liaison officer with the U.K.'s national crime agency he splits his time embedded with the FBI and at the British Embassy and Luke Dimbalski partner at Dibba Voice Plimpton and then next to Luke is Sean Newell he's deputy chief for cyber with the justice department's counter intelligence and export control section so welcome panelists so I'm going to turn it over to each of them we'll just go in this order to give a couple of minutes of who they are a little more detail than what I provided and kind of how they come at this topic so just a little bit of framing and then we'll get into the substance of the discussion Mick right so am I on? so I am the deputy chief for computer crime so really the limiting factors on there are computer and crime so I am dealing with all of your attacks and intrusions that are conducted for criminal purposes not necessarily every attack or intrusion that constitutes a crime because there's a division in the context between those of us who are prosecuting criminals for criminal purposes and those who are prosecuting nation states or terrorists and that would be I'm sure you hear more about at the end of the panel so all of the computer fraud and abuse act prosecutions in the country come to my office for consultation and we run an international capacity building program to help improve worldwide capability to fight cyber crime the iceberg part of my title as I like to call it is electronic evidence because it's not actually in my title but it's actually more of what I do I help federal law enforcement actually comply with the laws that regulate our collection of electronic evidence including from networks and then we try and take those two things what we know about how people are attacking computers and the laws that restrict the ability to monitor computer networks and we try and turn it around and be helpful to the private sector in what we call our cyber security unit I was the first head of the cyber security unit and I've now passed that responsibility to one of my attorneys so those are three of the big areas that I cover. Thanks Mick. Will. Thanks very much Steve yet my name is Will Ly and I'm from the UK's national crime agency and we lead on all aspects of serious organized crime law enforcement in the UK. I head our cyber liaison team over here in the US so we have a team of three going up to four people over here embedded in various different locations. Cyber crime is by its very nature international and the US is an incredibly important partner for the NCA in tackling serious organized cyber crime and yeah that's what me and my team over here in the US do. Hi and I'm Luke Denboski and my career in cyber started much like the work that these guys are doing I worked for 14 years as a prosecutor at the justice department a part of it with Mick in the computer crime section a part of it with Sean in the national security division two and a half years as our representative in Moscow to the Russian government on cyber and other issues and when I left in 2013 I like to say that I had absolutely all of that under control and I'm not really sure what has happened since these guys will have to explain that. For the past three years I've been heading the cyber and data privacy practice at a law firm I'm based here in D.C. the firm is global devil boys in Plimpton and so my teams are working on at any given time between maybe six and ten cyber incidents it could be an insider attack it could be a nation state attack could be something that came through a vendor or supplier so we're dealing with a wide mix of incidents all at the same time around the world and that's my job these days I'm Sean Newell I'm with DOJ's national security division Luke I think the issue for Russia is we're going to send you back if that's okay with you it's been nice knowing all of you so I'm in the national security division I oversee our portfolio of cyber investigations that deal with nation states and their proxies for those of you who don't know the national security division we are the department's newest litigating component we were stood up in 2006 following the September 11th attacks the idea of staying up in national security division had to do with trying to tear down that wall between criminal and intelligence tools and attacking the national security threat at that time the primary impetus was the terrorism threat but we've definitely applied those lessons and methods to a variety of national security threats including a national security cyber threat where we use both criminal and intelligence tools to eliminate the threat and figure out how best to disrupt them and kind of manage the criminal unclassified side of the house and the national security like sensitive techniques use the US intelligence community and things along those lines so we can all work together to counter this problem great well let's get started so generally speaking we're going to start with the cyber crime side so the pure criminal side and then we'll work our way through the topics to state activity and norms so Will's going to help us get started so Will can you give us a lay down on the state of play in cyber crime yeah well I'll do my very best so I thought I'd just run through and touch on a few points probably some just some emerging trends that we see in cyber crime at the moment which I'm sure hopefully some of these will be familiar to everybody in the room here so I think probably the overarching theme is we're seeing increasing levels of sophistication across the board cyber crime is getting more technically sophisticated by the day secondly and probably most importantly there's been a real low lowering of the barrier of entry into conducting cyber crime which in effect has meant that there's a proliferation of high-end cyber criminal tools and techniques around the world 10 years ago to conduct a cyber criminal scheme you needed to be pretty technically sophisticated you probably needed to be a hands-on coder nowadays you just need to go and know where to look in the right place and buy those things bring them all together and you can conduct a very sophisticated very impactful cyber criminal scheme so a really good example of that is criminal cash out a few years ago you had to have had a mulling network of people in the countries where you had to make fake cards and all that type of thing nowadays you can just pay some people to drop the clean Bitcoin into your account thirdly I think we're seeing increasing levels of collaboration something that I think I've definitely noticed recently is that actually lots of cyber criminal networks are all interconnected in a way that we probably didn't realize before and the reason that they're better than us cyber criminals collaborate better than us and there's reasons for that but yeah there's certainly connectivity in a way that probably I never used to appreciate and lots of these cyber criminal groups and networks are all collaborating with each other to an extent that I never really properly appreciated previously fourthly I wanted to touch on it hybrid actors I mean we're probably quite a good example of this sitting on the stage as we were split between national security and criminal to a certain extent and they're generally stove pipes in government and actually they're probably stove pipes in the private sector and elsewhere and there's these structural gaps in our system whereby we deal with something as a criminal threat or a national security threat and actually we're seeing lots of what I would call hybrid type actors that sit in the middle and that makes delivering a response to those particular individuals really different so what do I mean by hybrid threat criminal actors for example who have the support or the backing to some extent of a nation state and yeah that makes delivering a response to those particular cyber criminal networks really really difficult and that's something that we're seeing more and more in the criminal space and I'm sure the national security guards will say they're seeing it more coming from the opposite direction and then lastly on the criminal side I think Steve just asked me to touch on geographical areas I think we still see predominantly high end cyber criminal threats coming from Russia and former Soviet Union countries but as I mentioned earlier in one of my previous points the definition of those tools mean that cyber enabled crime crime that you used to commit West African fraud type crimes you used to commit on the phone or through mass mail is becoming increasingly easier and increasingly profitable to do online so you touched on some of the specialization that exists within the I've heard it referred to as the underground economy or the cybercrime underground network can you spend a little bit more on that you did mention as an example the cash out business service but there's a whole bunch of others can you touch on a few of those to illuminate yeah I mean you can go on to the cyber criminal underground and by basically anything I was describing it's a bit like that bar out of Star Wars I forgot what it's called now but you can get anything you want in there I don't need to have technical capability in anything I can go in and I can purchase a malware package of somebody I can get a service that will spam that malware to millions and millions of people around the world I can get somebody to check that my malware doesn't hit against antivirus vendors there's no point spamming malware to millions of people and they're getting blocked by McAfee and Kusplersky and all the rest of those common AV tools and then if I want to monetize the access that I get on victims well I'm just going to pay somebody to do that as well so a really good example in how this has proliferated is ransomware I describe it as probably one of the almost like it's getting on towards a perfect crime you know a lot of the areas that we used to be able to cash out was obviously a big vulnerability for some groups because if you wanted to cash out victims in the UK you needed a money milling network in the UK and it involved buying high goods and posting them to various places around the world and it's really expensive for criminals to do that nowadays I'll just drop some ransomware on your machine and just wait for you to pay me in Bitcoin and that's how this proliferation is getting is scary in a way and I would agree with all of that and I guess what I would say is I think it's been even more than just the maturation of the cyber crime marketplace like any marketplace that starts out build your own and then you find other people that know how to do something you don't know how to do and there's some collaboration and then there became carting forums where there were sort of closed communities that you had to know about and be vouched into in order to kind of enter the criminal underground the explosion of mass market crime as a service I think is a direct result of the emergence of torrent and services so now not only do you not have to be a technical person to know how to commit a cyber crime you don't have to be a technical person to hide yourself online and find the people who will sell you these services so we see places like Alpha Bay and before it's Silk Road having these mass bizarres and it's true a lot of what was on those market places are the sorts of things that are hard to buy legally like illegal narcotics but there was also a surprising amount of cyber crime as a service on those market places and it's really made it available to a mass market with a very low skill you lower that barrier to entry to becoming a cyber criminal to almost zero because it's very very easy to access these very public market places yeah and I think you narrow the gap of impact so a good example of that is DDoS attacks I think probably ten years ago there's probably only a few intelligence agency nation state type capabilities in the world that could knock a country offline and a few years ago some kids sitting in their living room in the UK did it and they're like 17 was that MRI you're referring to? yeah there was when Liberia, the entire country of Liberia got knocked off line and some other kids attacked a DNS provider and knocked a fair bit of the eastern sea border the US offline for a period of time as well from their living room in the UK and so you're seeing in narrowing that gap of impact that these individuals can achieve the proliferation of these high end tools and techniques wow so Mick so given the global nature and also the increasingly high impact of some of these schemes and capabilities are nations successfully working together to do what this issue? that's a qualified answer Steve I mean we are working together that's for sure we've got a representative from the UK sitting right here who's embedded with the FBI that's something in and of itself like I said my section has been pursuing international capacity building for well over 15 years where people including some of the people in the audience here who used to be part of my section would go out and train internationally to build the capacity in other countries the Council of Europe has been a leader in that as well but honestly even though we've had operational successes things like going back to when Luke was with CSEPS and did Game Over Zeus which there was a lot of international cooperation those operational successes really relied on individual relationships and our willingness to partner and I think the movement that I'm seeing is that countries are starting to realize we can't always just rely on the goodwill of another country we need the ability to effectively investigate international evidence using our own domestic tools right now if you want evidence from the United States there's somebody who has evidence at Google and they want that evidence in the UK they've got to come to the United States and ask for us to help and we do our best to help but we have to teach them concepts like probable cause that I'm guessing there are probably a fair number of lawyers in the audience today that find that a difficult concept themselves try teaching it to someone who doesn't participate in our legal system at all so we are reaching for ways to improve international investigations where countries have to realize I'm willing to give up some of my own sovereignty my own interests some of the control I could exert in order to make other countries more effective in this space so you've seen the United States do that we're actively negotiating at the Council of Europe for a protocol to the Budapest Convention the Budapest Convention is the leading multinational instrument that helps advance cyber crime and electronic evidence are there ways that we can sort of let another country use its domestic process to reach evidence in the other parties to the convention and there are now over 60 parties to the convention it's been growing rapidly in recent years and including in Latin America in Africa in Southeast Asia there are now parties from all of those regions of the world it really is becoming a global convention I guess the other thing I would say is some of you may have heard that not too long ago just over a year ago we passed the cloud act which is an incredibly significant new authority for the United States to enter into bilateral agreements to lift all restrictions at the Council of Europe we're talking about things like maybe they should be able to get subscriber information just to find out who's behind an IP address in the cloud act we would actually enable a partner like the UK to use their own wire tapping, pen trap authority to lift all restrictions because they have a system that has a strong human rights record that has adequate protections checks and balances, high standards for privacy and so why shouldn't they be able to use their own system to investigate domestic crime without the US law standing in the way so I think that's really the change that I'm seeing that has to do with the rule of law is sort of countries recognizing others authority that's fascinating so let's pivot a little bit into the role of victims in investigating and dealing with these issues we're talking about these investigations kind of in the abstract forgetting the fact that perhaps that there are major victims here and elsewhere in the world Luke can you help us to understand the role of the victim organization in responding to a cyber incident and what must they consider in cooperating with law enforcement to get things moving so that we can begin to figure out who's behind it and impose some consequences Sure Steve so it's complicated and not every victim has the same level of experience working with law enforcement if you're in the US financial sector you're pretty battle hardened at this point especially the major players they've been around the block they've been part of information sharing and analysis centers and organizations so they have an idea about how to cooperate with each other and how to cooperate with the government but even they struggle with these issues because to continue with that example they are heavily regulated and not all parts of even the US government have the same lens or approach when they come at them some may think of them less as a victim and more as a party that was derelict in protecting people's private information and so it's complicated for them there's a lot of fear and panic I guess for my role what's particularly gratifying is in major cyber attacks you are basically meeting them in their worst moment you don't hear the word existential too much for too many things I suppose terrorist attacks can rise to some level like that but cyber it's out there quite a bit as a word with our clients where if this really happened and it hit our entire network and took everything down and we were offline for days that might be the end of us and when they're thinking as individuals it certainly could take a lot less than that for it to be the end of my job and my career so they are afraid and it's a chance to help them think in advance ideally through these issues what does the decision tree look like people mentioned you know Mick and Will mentioned relationships and I really think that over the course of my 18 years around these cases the technology has always been interesting but it's always changing and it's in the background the bad guys have been interesting and they are changing to some extent and that's always there what has been a constant formula for success are relationships relationships of trust when I was in the CSIP's role or in the NSD role I would go to a place like the European Cyber Crime Center in the Hague on an operation like Silk Road or Game Over Zeus and we would sit down with 16 or 18 countries and it was all about how can we find common ground how do we trust each other with certain things and certain levels the company, the private sector victims are wanting to do the same yesterday I brought a major client of ours over to see Sean and the team they wanted to understand how would a national security threat play out so we brought the general council and their top reports in and they spent time with us explaining how they would work what kinds of things are people going to come in and raid jackets and take over our building are you going to issue a press release that blindsides us and makes us look like a fool in front of our shareholders, regulators and everyone if we share something with you how will you use it who else will you share it with if we are hit and we want to go out and do our own dark web activity or searches or gather threat intelligence where are the lines there what can we do and where do we cross it so these conversations are best had in a time of peace and not in the fog of war so that's a little preview of the victim issues how common is it that you're working with a client of yours prior to an event where you're advising them on incident response planning, developing these relationships is that the outlier that you just described or is that becoming more common for the more sophisticated clients it's becoming more normal and people are realizing they're in this digital Darwinism era where if you try to hunker down and stick your head in the sand it's not going to work out it's not a long time game so the first thing the GC said yesterday is we're a when and not if company you know it's going to happen to us if you got anything to sell any valuable data valuable intellectual property or anything you're going to be on someone's target list if you're not being targeted I think you need to find a more profitable line of business or something because you're on the menu you're on someone's target menu and it's a question of how high up the food chain you are as a target and so they came in with a right attitude if I can just add to that I mean when people ask how do I prepare for a cyber attack the first thing I say is hire former CSEPS attorney that's in the private practice now but the second thing is go to cybercrime.gov and read the best practices for victim response and it really does talk about building these relationships in a time of peace be ahead of it, do those things ahead of time and think about what information you'll be able to share with law enforcement before you're faced with that very vulnerable feeling of I've already lost control of this information I'm not ready to let anybody else have it even the FBI it's very hard to think clearly in a business continuity disaster type of scenario continue to pull this thread a little bit slightly out of order on my roadmap here so in terms of an effective organization managing their incident response process so back to Luke how do you recommend that they interact with some of these third party providers perhaps they don't have a huge incident response team internally maybe they're working with a company that does this for hire or maybe a threat intelligence firm and then also the role of firms like yours outside council how does that whole system work so I think the best prepared of our clients they have those vendors the Mandians, Crowdstrikes or whomever of the world they have them retained through us as outside council or their preferred outside council for really for two reasons one is attorney client privilege is most obvious because now when you announce your data breach at 4 p.m. you will be sued by 4 p.m. you will be sued in a class action in Florida 7th circuit or the 9th circuit 11th 7th or 9th you'll be sued by 4 p.m. sorry so if you aren't ready with a privilege mentality and an evidence preservation mentality that an attorney brings you're hosed to use a technical legal term the other thing is these are high speed excuse me internal investigations the prior generation of attorneys that came to this space were predominantly privacy and compliance lawyers they are very important and still needed we have them on my team but they were more ready to receive the facts from the forensic team and say okay you have to make notification in Iowa Tennessee and Belgium we'll send out the letters I don't want to demean them but their major was not how do we craft the forensics to prove the negative if we know that APT 10 doesn't really target PII in the 75 prior FBI investigations because we called these guys and they told us that because we knew who to call and what to ask we know that we can have a good position with auditors and regulators that they weren't looking for those droids on these servers they were after our IP that's their plan so understanding the threat space being able to speak about forensics and design an internal investigation to possibly create better legal options on the back end is what my breed of cyber lawyer does and then the privacy gurus can take that and say alright now with this more nuanced view on the facts we can see here and here where we have notification considerations and here are the risks okay so your mention of perhaps APT 10 and understanding what they're after is a great lead into this question which is attribution figuring out who is behind a particular act how's that are we doing better at that is that becoming easier or harder getting harder it got a lot harder on the criminal side when TOR became standard for a lot of crimes but even when we were dealing with proxies and there were proxies far before TOR the electronic trail could be difficult but we still had the money trail and I think a lot of cybercrimes in the past 10 years or so were solved through following the money trail and not the electronic trail and that is going away now going away now too with virtual currencies so when you have these mass market ways to anonymously transfer value it is enabled new crimes like ransomware and it has made investigations more difficult but I leave you with the impression that it's hopeless there's no need to report a cyber crime you can just hire Luke and feel like you can sleep well at night at that point because you've covered yourself the truth is that we are still solving significant cyber crimes just take a look at the last two weeks we announced one of the largest data breaches in history was conducted by a charged individual in China the Anthem Breach we took down one of the largest referral services that were referring people to the dark net and were taking money according to the allegations in order to drive traffic to illegal narcotics sites and all the way across the Atlantic in the Hague at the European Cybercrime Center it was announced that there was a group of six different international bodies or countries that collaborated to take down the Gosnym criminal network so you now have cyber criminals who are being prosecuted not only in the United States and we still are prosecuting a significant number of them in the United States you're seeing cyber crime prosecutions successful ones and ones with significant deterrent value in countries like Moldova like Ukraine like Bulgaria so these are places that really weren't prosecuting cyber crime to the same extent 10 years ago and yet today because of capacity building they're able to accomplish these things on their own that's great Will, do you have anything to add to that? Yeah, I would agree that absolutely I think attribution is getting more difficult in general I'd say there's a number of ways that we come by that collaboration is clearly one of them data is incredibly important really, really high in cyber criminal might not have been such a high in cyber criminal be careful 3, 5, 10 years previously and then overlaying humans with what we know human intelligence what can people tell us what can the lower level individuals tell us in this and so as the scale of this gets bigger attribution in general gets more difficult but actually we're maintaining an operational tempo it is still possible in this space and delivering operational results is still very achievable Great Well, Sean's been quite quiet to fill in the panel up to this point but it's time to pivot to him so far we've discussed cyber crime but recently the Justice Department's been making news by seeking criminal charges against individuals acting at the best of a foreign government so Sean what's the thinking behind this? Thanks Steve I would actually say it hasn't been that recent now it's been 2014 when we brought charges that's five years ago it's not too recent in my mind so we've been doing this for a while I think back then I'd say pre-2014 we brought our first case against nation state actors we charged five members of the People's Liberation Army with among other crimes intellectual property theft through hacking prior to that time I think the bureau and the department had gotten really good at figuring out what these actors were up to and watching and figuring out their modus operandi the issue was that is very valuable and I think that's a traditional CI approach to a national security threat a counterintelligence approach to a national security threat at some point in time I think leadership rightfully so said okay the traditional CI approach if you watch the adversary learn what they're doing figure out what their network is wait for them to go visit a dead drop in the middle of the night and then maybe figure out who their sources are was not working in the cyber realm because it sounds cliche but people are things were happening at the speed of cyber and so you would watch these actors and figure out who they are by your intelligence collection methods and while you're watching them they're sucking out millions of dollars some even go higher than that of intellectual property and other valuable information from our private companies clear defense contractors and things along those lines so pre-2014 leadership aside there had to be a switch we'd still take that CI approach and it was still necessary but we would be better at figuring out when to switch to a disruption and deterrent effect operations and so one of the methods because the department of justice in the FBI or law enforcement agencies was thinking we should be bringing the rigor and intensity of our criminal investigations to the CI threat and using that as one of the many tools to disrupt and deter these actors so that's what we decided to do number one we'll try to arrest these hackers granted when they're either intelligence officers or working for an intelligence service from effectively a state haven in their home countries we do have a very low chance of arresting a lot of these actors and we recognize that but we felt that there was still a disruption to turn effect from bringing criminal cases or just at the very least taking an investigative approach a criminal investigative approach to these actors and there's other ways to disrupt them and the idea behind that is number one you give you can test the space you don't make it a zero cost approach to these hackers which it kind of was spearfishing emails and entities or trying to watering whole attacks and things along those lines with very little risk of being caught identified or disrupted and so now we're just raising the cost contesting the space it gives the network defenders some breathing room it we help pitch information from our investigations to those network defenders so they can better defend themselves but with regard to the adversaries and their decision making calculus you don't have to insert in their minds am I truly as anonymous as I think I am will my future be as carefree as I hope it would be can I travel these places when I want to maybe if I want to give up being a hacker someday and go work for a reputable company can I do that these are all things that we're now putting in their minds and hopefully as part of other efforts we'll discuss later on starting to change that cost benefit calculus and having a deterrent effect with a lot of these adversaries so what is the relationship between the types of cases you're charging and the norms that we heard about earlier either from the 2013 or 2015 group of governmental experts report and then also kind of related to that are we concerned that other nations will want to criminally charge US government employees that may be doing things that violate their domestic laws so what's the relationship there between what we charge and expected state behavior right so if you actually take a look at a lot of our cases that have been public that we've done out of the National Security Division working with US Attorney's offices around the country you actually notice that a lot of those cases probably track many of the norms that have been out there through a group of governmental experts and what not or through other international kind of consensus building entities so for example we have had a lot of cases targeting the theft of intellectual property for the benefit of foreign foreign companies by state sponsored actors that's something that I think we in the United States have been talking about for a long time and a lot of our partners and allies have for many years now been saying the same thing which is you know granted espionage it's traditional tool statecraft it's everyone does spy I think you know President Obama said at the time but what we there we think there's certain red lines as an international community that should not be crossed and that was one of them you should not steal intellectual property for the benefit of your own your own companies or state-owned enterprises things along those lines so you see a lot of our cases that are coming out publicly do tend to track those norms other cases you know vows should not attack critical infrastructure critical infrastructure in peacetime we have several cases where we have actors I can think of the Iranian DDoS attacks against the financial sector that were done on behalf of the Iranian sorry the Islamic Revolutionary Guard Corps same thing these are types of cases where we're saying that is the type of activity that we believe countries should not engage in in peacetime and you'll see a lot of our national security type cases track that to help reinforce those red lines both with our partners and in our adversary size and in which case we're making a policy statement that the U.S. does not itself do those things yes absolutely so that brings me to your second question which is you know do we have a fear that our own operators will be on the receiving end of criminal charges in other countries so I go back to the point of what I said I think that there is all countries spy that's something that is normal and we hope that our countries are gathering intelligence to better protect ourselves down the road but what our intelligence agencies do not do are those types of things that we are saying other countries should not do which is attack critical infrastructure in peacetime steal intellectual property from a let's say a Chinese aerospace giant and hand it over to Boeing and they do not do those things and so I think we tend to stay on the right side of those lines or what we believe are the lines but at the end of the day personally I have a hard time buying the fact that foreign governments or adversaries maybe are not charging our operators out of some sort of trying to recognize norms I have a feeling if they can identify the adversaries they would engage in their own cost benefit analysis and charge them regardless of what the Department of Justice and the FBI are doing so we are on the little bit of the home stretch for my guided discussion that will pivot into question and answer but a couple more things so back to Luke for an organization that has been the victim of what might appear to be a state sponsored attack are there special considerations that they would have in mind or need to be able to navigate in the incident response phase that would be different from if it was a pure financial motivated criminal attack and especially if you're working with the Justice Department and there's discussions around whether your client would be willing to be named in a public charging document as being the victim of a state sponsored attack a lot of considerations one is do you have cleared personnel people with clearances or access to them to be able to go into classified space and find out a little bit more whatever the government might share about the nature of what's hit you you may be served with national security process normally as a victim the approach is a voluntary one as far as that can go but it's conceivable that your client could receive what's called a FISA warrant or issued pursuant to the foreign intelligence surveillance court or an NSL national security letter which is the rough equivalent of a criminal subpoena and there are specific rules about who you can share these with and how you respond and you can't just run off to your board and tell everyone in your HR department about this you could be crossing legal lines there there's some sophistication required in preparation for the response to this in terms of the media aspect and what you're asking about Steve it is a big consideration to have your client named in an indictment that points a finger at a major foreign country because you may have business operations there or dealings there and the retaliation risk could be high and so a lot of our clients are very worried about those issues they want to get justice so to speak or at least make it better for their stakeholders they want to do the right thing as a participant in critical infrastructure and broadly from that standpoint but they're also worried about the collateral damage and retaliation risk just on the going back to the question about attribution and you talk about whether you as a company have been hit by a particular nation state I think when you suffer a major incident as a company that could be a national security or criminal risk out there I think it's on that attribution perspective it's taking a longer period of time to figure that out and actually with the emergence of these hybrid actors could you have actually been the victim of both at once and how do you deal with it as a business are you going to be subject to national security process or criminal process or you know actually or mixture of both I think that's getting a little bit more complicated I think those lines in the road are no longer as clear cut as they used to be right okay well I'd like to wrap this up maybe back to you Will with the final question that Sean made a pretty compelling case for why the U.S. has taken the attack that it has with trying to deter malicious cyber activities by states with criminal charges what are some other ways that we can achieve deterrence in cyberspace I think there's an enormous toolbox that we have at our disposal here to kind of like counter that maligned behavior and that can really be anything that associates a cost to that particular piece of activity so for me it's not just about delivering specific tools like indictments or U.S. treasury sanction designations it's about two things firstly it's about using a blend of those tools all at once so making your response multifaceted and secondly it's about doing that in a really collaborative way we are finding that we are conducting all of this activity as coalitions of people that all believe in these types of norms in cyberspace so I don't want to go into particular types of ways that we can counter this but I'd say that actually as a community moving forward we're using a blend of those different tools all at once and we're doing that in a much more collaborative international way if you don't do that I think we've found particularly in the criminal side that you risk potentially driving an evolution in the sophistication of the adversaries tactics if you hit them with the same thing every time they will evolve to notify that tactic that they can use against you Sean quickly and then we'll move on to questions I think that's a fabulous point I just draw some public examples of this recently we've had two instances recently in the national security context where we've had more of a coalition of international an international coalition of like minded countries have come out to speak about some malicious state of monster behavior in cyberspace the first one was in October 2018 there was a Russian operation to go after anti-doping agencies and to put information about athletes out on the internet somewhat was kind of tweaked in a way to make it look worse than it was and that was through a cooperation of I think the United Kingdom Canada the RCMP were a very important partner in there and also the Dutch military intelligence you know go back two years ago you would never have seen that type of cooperation in the national security threat especially with regard to a foreign military intelligence agency cooperating with an American law enforcement agency to counter one of these threats and I think it's a really good, useful example and then more recently the APT-10 in Diamond Sur probably about 12 or 15 countries that came out together with statements not just the Five Eyes, the US, UK, New Zealand and Australia but like other countries in Europe and Asia that came out said this activity emanating from China is unacceptable and we think that it should cease and so I think that's a very positive trend Great, great point. Okay, so we have probably about 10 minutes for questions and we have microphone holders and so because we are being live-streamed and also so the folks can hear please do use the mics so I see a hand here and if you'd make sure that you do ask a question and not give a speech and then we'll have one of our speakers address it and then we'll move on to the next question so we won't all five try to answer your question I guess I address this primarily to Michael in terms of law enforcement response to sophisticated cyber crime, what are some of the investigative tools that are most important in solving these crimes? So if I can change your question just a little bit it's not so much the investigative tools it's what do we have to do to make a difference and make an impact on people not doing this crime I think what we need to do is solve a lot more than what we are now I'm not a criminologist but I'm starting to be educated on some of the literature that says that the certainty of punishment is a lot more effective than the length of punishment I've put people in their 20s before the court to be sentenced and they've received 10, 12, 14 years I'm increasingly convinced that that may not have been the greatest deterrent message we could have given rather than have 10 or 15 of these individuals before the court so that they were all being punished for somewhat less amount of time the problem is the easy cost effective thing for congress to do is increase penalties and that shows they're serious about cyber crime I think to be serious about cyber crime we've got to have a lot more cyber investigators we've got to retain the experience once we have maybe even with increased pay and we've got to and we've just got to have a lot more of them great I see let's see I think you had your hand up earlier right so we'll come back here this is a quick question what about if a victim got an email and the sender says you know I have your password and your password is the following and if you don't give me $100 I'm going to use it to steal your identity what should be the answer or the reaction or what should the victim do in such a case can I ask our moderator to step into the world because I know the FBI has guidance on when you're being extorted over the Internet we typically don't recommend paying extortions that's the official line if they say they have your password I would immediately seek to change all your account passwords on that account and any other account that may resemble it but and then also report that report that incident to the Internet Crime Complaint Center because we are aggregating a large number of small individual incidents and losses so that we can put together larger cases but I guess it depends whether the person is offering proof that they're in your account or whether they're just blowing smoke but definitely have good practices on your two-factor authentication if your email service or brokerage account offers it, use it it's quite a great counter measure for preventing that sort of thing but we don't recommend paying ransom because you're fueling the very activity that we're trying to prevent and let's go to Mary and then we're going to come here next Hi, I'm Mary Greer I work at the ABA Rule of Law Initiative and I wanted to follow up, Mick, on your observations on the evolution of technical assistance at least over the 15 years that CSIP's has been providing such great services do you have any reflections for us and I want to include all the panelists on what the constellation is what's been more effective than not typical training models capacity building the multi-agency approach and Will had made the good point that it's beyond the enforcement rubric and I'm wondering again within the confines of rule of law technical assistance other things that we should be doing or thinking of doing or stop doing I think the first thing that I would say is that I do think that capacity building is incredibly important in fact the State Department has been wonderful in using some of the money that they're allocated to actually fund Department of Justice attorneys to go out and do that we've started with one or two here or there mostly focused on intellectual property but we have just created the first global law enforcement network of international computer hacking and intellectual property liaison prosecutors that's a lot of words but we just call them the Glenn of iChips and the Glenn of iChips is going to be a dozen federal prosecutors fanned out across the world in Panama, in Brazil in Croatia including one that will be stationed at the European Cyber Crime Center specifically to address Eastern European transnational organized cyber crime working collaboratively with the collection of the European police and prosecutors who gather there in the Hague so as far as what we should be focused on I think one thing is we tend to do more comprehensive training across all of the necessary levels investigator, prosecutor and judges I think judges are the ones getting the least amount of training right now I think investigators actually probably are getting the best training getting some training and we hope to improve it significantly with the Glenn of iChips but judges are getting less training and ultimately all of these cases if we're successful are headed to a court so I think judicial training is critical on the capacity building front I mean lots of different countries lots of different agencies around the world have a range of different really really excellent capacity building programs one thing that I think we're probably not very good at however are the NCA in a particular country providing a forensics course that the FBI or the DOJ provided to a similar group of people two years earlier so that's what I say on the capacity building front and secondly it's not all just about enforcement like cyber crime is the one crime of all of them that you're not going to arrest your way out of this problem and actually 98% are going to arrest themselves by just undertaking some really basic cyber security hygiene, cyber security measures and actually that remaining one or two percent are going to be victims if you have good cyber hygiene or not so we should be utilizing operational success is better to put that protect message out there so yeah the two halves of the of the same coin there Thanks Will, okay so we're coming here Hi one question that I had for both on the criminal side and the nation state side is this question of commercialization so clearly it's reducing barriers to entry for conducting cyber crime as well as nation state attacks but do you see this in your work as introducing any sort of opportunities for you to improve your investigations or to better hold people to account so at Citizen Lab when we're tracking nation states targeting dissidents in civil society commercialization is sort of very helpful in that once you've seen one instance of an attack or one actor using a commercial tool it's much easier to map out the entire extent of that activity and uncover other attacks Sean do you have any views on that? Yeah I think you raised the advantage of commercialization for being able to figure out what's going on and who's doing what but again I think it kind of goes back to the point made about cyber criminal underground and these people who established their niche these companies are now enabling I think foreign intelligence services or law enforcement services all around the world to get into the game of cyber investigations and you know some of them just don't have the same rule of law protections that we like to have that we have here in a lot of other countries around the world I think that's where that friction is posing problems significant problems. Yeah I think it's really interesting I mean it allows the various acts as whether you be a criminal or nation state to go in and do what you want to achieve at a low risk and a low cost but at the same time it presents some sort of opportunity to us as law enforcement or investigators across the pieces and actually if you have visibility on a criminal service that they all utilize then obviously that's a real investigative advantage. Okay I think we have one more question in the back I saw a hand earlier yes you and then this will be it. This is a follow-up question on ransomware there was an NPR piece this morning on Baltimore and the whole city being shut down by ransomware and Atlanta being hit earlier Baltimore does not have the resources to deal with it how do you they're even looking at possibly going back to paper for house transactions what recommendations do you have for cities with limited resources? Well ransomware is a huge huge problem and we are seeing that which would be presumed to be a cyber crime offense maybe that would be aimed at the normal individual starting to hit critical infrastructure owners and operators, towns that sort of thing. DHS and FBI have put out ransomware advice out there on U.S. and other websites I know with Baltimore our folks are working on that with the city it's a major problem and organizations need to be fostered to be able to deal with that issue so it's hard for me to diagnose that from afar anybody else want to weigh in on the problem of ransomware? I would just say there are many many resources out there some of them are more expensive than others but certainly one that is heavily involved when there's something like that happening is the Department of Homeland Security it's less of the law enforcement concern where we are concerned about trying to find out who did it but it is DHS that if you want to give the analogy to the policeman and the firefighter they're the ones who are trying to assist in the recovery of an event like that and so really it's our Department of Homeland Security I think that would be most involved just one word on that from the private sector experience is you know preventing it segmenting your network all those things but the real coin of the realm for our clients no pun intended has been ability to restore from backups you know and if it's just a mirrored rolling backup that's going to be encrypted as well so it's expensive to store a snapshot and to keep a periodic snapshot but when our clients have the snapshot they can restore back to that snapshot and they can come back and they'll lose maybe some days or hours and that's expensive for a government to think about but the alternatives are far more expensive to me so I think maybe it needs to be an investment in that along with all of the hygiene and segmentation steps cleaning up RDP connections dealing with phishing and trying to raise the game on all of that from a prevention standpoint but having the backups is a fallback Paul thank you very much panel and thank you Mary and Linda for 20 years and suddenly everyone's listening we care a lot about internet freedom we care a lot about openness we care a lot about privacy we care a lot about security and that creates interesting tensions and we are here today to talk about some of those approaches kind of the considerations that people are taking into account and just how we should be thinking about this as we go forward in order to create the internet that we all want so we will and then we will have a conversation and think of questions we will open up for Q&A maybe 10-15 minutes before we wrap so first off we have Dr. Andrea Limago who is the I want to get this right Chief Social Scientist of Virtue and thinks about the social side of these technologies how do we insert the people we've been on a panel before that ended up being absolutely fun because we were there to talk about cyber security and we all ended up talking about people which is actually I think the right thing to do so go ahead thanks very much it's great to be here and so like Heather said we're going to a software company that focuses on data privacy and encryption and the technology is very important obviously but the interaction with humans as well part of my work with a background in global relations in conflict studies is how some of those technologies fit into the broader global trends that are going on and one of the big areas that I've been focusing on recently for the last few years is the rise of digital authoritarianism and so over the last 8 years internet freedoms have been on the decline I feel like freedom of the net from freedom house based on their metrics and so what that entails a variety of means and I'll talk about some of those how governments are using these tools and then how some of those tools are spreading to non-state actors as well but basically it was tightening the control over users data I recently saw in our article a few days ago the hegemony on communication and I kind of liked that notion because that is what they're trying to do complete information control really by whatever means possible and a lot of it is some very creative means that just weren't foreseen even several years ago and so the way that this is manifesting is everything from elections which we hear a lot about in the US but to be clear the election interference via digital means is only one area that's getting the interference and that's what's going on in over a dozen countries in 2016 so it continues to expand in that area other areas in addition to we think about the cyber tax that kind of at least seeped into public consciousness as far as what's going on to infer and jump privacy but there are other areas that are going on in addition to some of the more cybersecurity issue areas that go into the legal realm and the realms of data localization and a lot of those laws are increased which basically requires local data storage in those countries and then with some of the different regulations as well also entails government access to that data when they want to you know they can say under only certain conditions of data they can demand it and so that's led to issues such as Apple storing some data in encryption keys in China it also has led to in Facebook the recent manifesto saying that we're not going to be adhering to some of those localization laws of authoritarian regimes and so there's a big discussion going on both amongst governments and amongst tech communities and businesses and corporations for how they're going to respond to those localization laws and it's not just China and Russia and this is something that we need to be really clear about across all these mechanisms from cyber tax and cyber crime that we just heard about the proliferation to non-state actors of these tools really is just it's expanding significantly and so the same thing happens when it comes to data localization Taiwan and Thailand and Vietnam are two of the more recent ones that have been passing various laws as far as either government access to that data or government storage of the data and both of them especially the Vietnam law really mimics a lot the Chinese laws that you see and so you do start seeing the rise of the rest going on across the globe when it comes to digital authoritarianism and so what that also is leading to is something called the splinter net or balkanization of the internet so instead of the global a free and open and global internet based on internet freedoms we're really seeing it breaking apart into national cyber sovereignty kind of internets that are going on across the globe and to take it to the extreme we've got the Great Wall the Great Firewall from China but they also have Russia recently passed a law to try and ostensibly cut off access to the rest of the globe Venezuela is considering similar legislation right now and so these are the things that what happens in China doesn't stay in China what happens in Russia doesn't stay in Russia these are expanding and so far there's been a bit of a vacuum on the side of democracies to push back and whether it's I know Norm was discussed about this morning the cyber norms has fallen apart a bit at the governmental level there's still some efforts there but at the UNGGE progress was being made and then fell apart and so some of the tech giants are starting to try and fill in and so absent democracies doing more some of the corporations are actually trying to fill in that gap to create the appropriate norms and the appropriate behavior of what to do in cyberspace and how to preserve security and privacy and so that's I think an interesting trend that's going on as well and just as far as on the one hand a lot of what my work does seems to focus on some of the doom and gloom that's going on from the surveillance society and the spyware and everything along those lines and we can't be naive to those threats and I used to work in the DOD I'm very familiar with the threat landscape that's going on but at the same time we also need to be aspirational and try and still build the internet that we want it to be and try and push toward that vision of the free and open internet and for me and I can get into some of these more later I think it's a combination of the technology as far as usable security and making it much more usable and accessible for people and just raising greater awareness and we can also talk about some of the bad information that also comes out of the tech community as far as what's secure and what's not but just following some basic security best practices can really help out 90, 95% of the people at a minimum but in conjunction with some of the norms that could be pushed forth and then some of the privacy laws we can talk about GDPR aspects and what's going on in the United States right now is we've really left a vacuum globally as far as leadership and establishing what the guidelines should be for data privacy across the globe and so we've got California coming out with their law that's at the farthest that we've got in the United States different states again are coming on doing something either copycatting or doing their own version it's very much a piecemeal patchwork is usually the analogy given and so we really do need some U.S. leadership then where we can push forth for data privacy law at the federal level to create some efficiencies and innovation and really protect data privacy also can serve as a model for other countries around the globe great, thank you next up Bill Marzak is a research fellow at Citizen Lab the co-founder of Bahrain Watch and a postdoctoral researcher at UC Berkeley thank you so unfortunately I didn't have the foresight to print out my remarks I'll be reading them off of my phone it's very modern I'm not checking Facebook or anything like that so at Citizen Lab I do work looking at commercial spyware so these are companies which are selling products to law enforcement to intelligence agencies to essentially hack into phones and computers of targets and the way that law enforcement sort of talks about digital investigations these days is they say there's a going dark problem and this sort of spyware industry the surveillance industry is offering the solution they say well, yes, there's the going dark problem everything's encrypted so what do you do? You need our products to hack into computers and phones of your targets and see what's really going on bypassing the encryption so the companies spend a ton of money on research and development finding ways to get into the latest iPhones the latest computers running the latest software and typically the way that this works is that they'll send some email or some message containing a malicious link or attachment the target opens up they're infected and then everything on the computer or phone is subject to surveillance including encrypted calls, encrypted messages as well as being able to turn on the webcam and microphone of the device in order to spy on activity in the vicinity and you probably also heard from the tech news a couple weeks ago this crazy story of the WhatsApp hack where your phone could be hacked by receiving this missed call from a malicious a malicious caller and what was happening there is even if you were asleep let's say they could send you a missed call the call would ring for three times and if your phone was on silent while you were asleep you wouldn't hear it and the call would not show up in the call log afterwards so it was sort of like the perfect the perfect hack and one of the components of this story is that there was a human rights lawyer in the UK who was targeted by this and the reason he was able to notice this is because he was sleeping with his ringer on so he would awaken to these weird missed calls from unknown numbers and his phone would ring and then it would disappear from his call logs and he questioned who am I dreaming what's going on here but it turned out that at times on this a leaker basically a spyware salesman from inside NSO Group collaborating with NSO Group felt he had been stiffed on an earlier deal and didn't get his commission so he decided to leak some details about how this surveillance works and the company the spyware company involved was allegedly this company NSO Group based in Israel that sells this sort of spyware around the world and the tools themselves they just sell it to governments who operate the tools but interestingly enough this human rights lawyer is sort of famous for filing a case against NSO Group so he was suing NSO Group at the same time he was being apparently spied on by the company and of course NSO isn't the only player in this space you've probably heard of others like Fin Fisher, Hacking Team, Cyberbit and this is way more powerful than surveillance getting wiretaps because a wiretap you hear what someone is saying and you get access to the phone whereas with this sort of hacking you get access to the phone and what do you have on your phone it's your whole life it's your calendar your transportation with Uber and stuff like that your emails it can even turn on the webcam and microphone to create new data and exfiltrate that and the common refrain among these companies of course is well yes we know this is very powerful technology we sell it only to governments and only to fight terrorism to identify those sorts of crimes but not surprisingly a lot of repressive governments are interested in this stuff too so we've seen governments like Saudi Arabia UAE by this sort of technology countries that define terrorism like Saudi Arabia to include criticizing Muhammad bin Salman that's considered terrorism according to the 2017 terrorism act there so this is obviously an area that's very ripe for abuse in terms of these surveillance tools and the companies of course like to tout the lawful uses of their tools but one interesting case is they said oh well our tools were used to capture El Chapo in Mexico but what we found actually is that so El Chapo one of the things that came out of his trial is that he paid a hundred million dollar bribe allegedly to the former president of Mexico and interestingly when El Chapo's ordered a hit on a Mexican journalist and he was killed in the weeks afterwards his wife and his colleagues began getting targeted with the spyware that the government was using particularly interesting in the context of the bribe you know there's a bunch of other cases we've documented at CitizenLab including one of Jamal Khashoggi's close contacts who was being spied on by apparently the government of Saudi Arabia so here's a question some of you might be wondering is this guy Bill how does he know about all this stuff from this crazy world of government surveillance and intelligence agencies it's supposed to be private these are supposed to be actors that are operating in the shadows well as I was saying earlier in many cases the way that this spyware works the way that you get hacked is you get sent something that you have to interact with in order for the hacking to happen like a text message or a link or something like that the WhatsApp thing is sort of very rare so the whole enterprise here relies on the people who are targeted not realizing that a suspicious message is actually malicious because if they do then they can forward it to someone like CitizenLab or someone like Amnesty International or EFF the electronic frontier foundation who are all sort of operating in this space and then what we do is we can actually analyze what we've been sent in fact there was a famous case in 2016 activist in the UAE Ahmed Mansour got some suspicious text messages prompting him to click on a link about new secrets about torturing UAE prisons I think was the title which is kind of odd to get that from an unknown number so he forwarded it to us at CitizenLab we clicked on it and were actually able to get a copy of the spyware and a copy of the bugs that the company was exploiting to break into the iPhone so we just closed that to Apple Apple released a new update back in 2016 iOS 9.3.5 and of course when the bugs are patched the spyware as written no longer works so this is a way to sort of pressure these companies because once this is disclosed all the customers for the company start saying wait a minute the spyware doesn't work anymore we paid all this money for our system what are you doing so there's a bunch of challenges in doing this sort of work getting in touch with the right person at the right time but this sort of field that I was mentioning when I was talking about the cyber security researchers as I'd call ourselves are looking at this field so it's clear that this is sort of a wild west type atmosphere you know you have governments doing deals with shady companies doing deals with shady governments who are then targeting activists and dissidents one of the subtle points here that I want to bring up though is that there's sort of two markets for these companies legitimate players so we know that some of the spyware is purchased by legitimate law enforcement agencies like for instance the Belgian federal police or the Swiss purchased spyware from NSO group in addition to other governments like Saudi Arabia and Mexico so the surveillance industry tries to cater to both of these types of clientele but it's a bit awkward because you have cases where the abusive customers the reckless customers like let's say Saudi Arabia UAE, Mexico are sort of sending this stuff out to civil society who forward it to us we disclose the spyware things get patched and that hurts both the legitimate customers as well as the illegitimate customers and so one of the things I want to talk about is like alright so that you can think of that as sort of a form of regulation as a last resort right like we're trying to send the message well if you know you target civil society your things will get disclosed and then you know bad things will happen for all of your customers so maybe you want to ensure that doesn't happen but what's a better way to think about this sort of regulation well so the sorts of spyware that we research are called intrusion tools or are classified as intrusion tools under the Vasanar arrangement for export controls meaning that governments who are party to the Vasanar arrangement should hopefully seek to regulate these and the regulations basically will if you want to export it we've got to sign off on it first and the problem sort of here so far has been you know we've had cases like for instance the government of Italy issued a license to the famous spyware company hacking team that said this is a global export license you can export this anywhere in the world and we don't care and you've also got cases like in Israel where NSO groups second customer their spyware company their second customer was the UAE and this was a big scandal apparently according to reporting in Israeli media that Israel issued this export license for the UAE a country which doesn't have official relations with Israel and of course there was recent drama in the press regarding NSO group in Saudi Arabia where it was alleged that some of the outreach from NSO group to Saudi Arabia was done without approvals from the export licensing agency in Israel so there's also been a you know for those of you in the Vostner debate there's a number of areas of pushback particularly among cybersecurity defenders who say well you know you're regulating these intrusion tools and bugs which is great but the problem is that cybersecurity defenders in other words the good guys who are researching this stuff often need to send samples of the spyware or things like this across borders and we don't want to apply for export licenses for that and then you've got problems with industries who say well yes we love regulation but you know we don't like this we want a level approach we want a level playing field and the same set of rules for doing business everywhere in the world so in another frontier in the regulation here has been trying to hold governments to account so like the government of Saudi Arabia maybe the government of Bahrain the government of Ethiopia which are using these tools there was a case right here in the DC circuit Qadani versus Ethiopia just a bit more than five years ago the evidence was pretty much a slam dunk in the Ethiopian dissident in the DC area who was monitored with spyware sold by a company Finfisher apparently to the government of Ethiopia and the spyware of course is supposed to be super stealthy and leave no trace but there was a hiccup in that this guy's Skype calls were being recorded and temporary files were written to his hard drive showing evidence that his Skype calls were being wire tapped so he sued the Ethiopian government under the wire tap act because there's a private cause of action there and of course the Ethiopian government was like oh well yes if there was a tort here it was committed entirely within Ethiopia if there was a tort here it was not committed entirely in the United States and there's the foreign sovereign immunity act and the entire tort rule that says that if the entire tort doesn't occur in the US you can't hold the foreign sovereign liable there's ongoing cases in other countries like the UK where a UK law firm is suing Bahraini activists who are spied on by this company FinFisher and in that case the UK precedent seems to be a bit more favorable because they don't have this entire tort rule but these cases are rare it's kind of hard to get the right kind of evidence at the right time and I think there's been much less success in holding the firms themselves to account so I'm looking forward to talking about more ideas of that once we get to the questions. Awesome, last but not least Robin Greene who is now a Privacy Policy Manager at Facebook to talk a little bit about how Facebook thinks about this and maybe generalize a little bit about the industry that seems fair to me and protecting users and their relationship with law enforcement so thanks thanks and sorry I'm not used to these mics are intense so thank you for inviting me to speak to be able to talk with you all today so when I think about public safety issues and I think when Facebook thinks about public safety and law enforcement access issues in particular we spend a lot of time thinking about exactly how we started this panel thinking about people thinking about the individual Facebook started 15 years ago as a pretty small community of college students who were just connecting to get to know each other to stay in touch with their friends and everything like that and in a little over a decade we have grown into a community of platforms that serves over 2.7 billion people around the world and so when you think about the size of responsibility that we have in order to make sure that we can fulfill our mission to help people build communities to help bring people together and give them ways to share and to organize and to start businesses when you think about the mission to allow people to communicate with one another and to bring the world a little bit closer together the only way that we can do that is by fulfilling our immense responsibility to keep our users safe and that means fulfilling our responsibility to protect the security of their data to protect their privacy and from arbitrary interference with government and to make sure that we are protecting public safety as well and so that sometimes puts us in the uncomfortable position where we are having to work with law enforcement while at times having to push back against law enforcement sometimes the very same law enforcement when they're requesting users data but that's also exactly the thing that animates our decision to move into interoperability and to end encryption across our messaging services to make a public statement as we did a few months ago that we will not locally store data pursuant to any government's mandate in a country that has a bad track record on human rights where we're not confident that we can keep those data safe what we do to make sure that we're protecting privacy and enhance public safety at the same time is apply a really rigorous process for responding to law enforcement demands and so what this means is we basically respond to law enforcement in several ways but in two primary ways the first is in response to lawful requests and because we are incorporated in the United States and in Ireland the way that we wind up responding to requests is if you are from the EU you're from an EU government you're asking for users data you have to go through Irish law that are incorporated and if you're from the US or from outside of the US and you're looking for the contents of users data you have to go through the US judicial process and if you're not from the US government you have to go through the Mutual Legal Assistance Treaty process which essentially enables you to obtain a warrant based on probable cause by working with the US Department of Justice as long as you can to obtain that warrant and then we receive that warrant in our compel to hand over the information but what this means is we receive requests for information and we scrutinize every request so whether it's for contents or non-contents and no matter the country we look to make sure that the request is lawful that it's appropriate in scope and that it's appropriate in purpose and that it complies with international human rights norms but if ever those elements aren't met we will push back on their request the first thing we'll do is go back to the government and work with them to see if we can clarify what they're looking for and see if we can identify a way to make sure that if it's a legitimate investigation and they really need the information can we figure out a way to scope their request in a way that is lawful and properly scoped and in compliance with international human rights norms we will push back and we'll refuse to comply or we will take it to court where possible and challenge the request judicially the second way that we primarily wind up giving data to law enforcement contents data is where there is an imminent threat of death or serious bodily injury we think that it's incredibly important that we contribute to public safety so if we see information on our platform that's suggestive of that kind of imminent threat we want to make sure that law enforcement or the appropriate authorities are able to act on it and so that's the other main instance in which we hand over users information to law enforcement but I think when it comes to increasing requests and the changes of technology and how most information is digital now we wind up seeing certain pressures and so Facebook in particular and tech companies across the board face a lot of pressures around issues like cross-border data requests and technical assistance requests surely with our move to end to end encryption across all of our messaging services we'll see increased pressure on both of those fronts but this doesn't sort of like hinder our resolve we are fully committed to deploying encryption across our messaging services and to making sure that the people who are dependent on these technologies for their privacy, for their cyber security sometimes for their freedom and their personal safety are able to enjoy their benefits but I think there's a lot to talk about when it comes to challenges with cross-border data requests and technical assistance requests and certainly we're facing those challenges all over the world so I'd be happy to dive more in it during the discussion Excellent so I think that actually that's a really interesting set of things to launch off our discussion with and what was really striking to me Robin and especially is thinking about how you balance these competing interests because I don't think anyone up here would say that we are against law enforcement and I don't think anyone up here would say we're pro-surveillance either and what's Bill, what do you think the appropriate balancing test is because I think that you will have having seen these things used improperly how do you decide what is proper and what is improper with those balancing tests and to what extent do you think about it globally versus kind of regionally or nationally well that's a great question I think that clearly we have a lot of experience in enumerating what is sort of inappropriate so targeting activists or journalists simply because of what they say rather than because there's any sort of real crime going on there other than freedom of expression I think though that there's sort of when we're thinking about regulating or thinking about targeting the companies themselves I think though the question is well picking customers and vetting customers before they're doing the sales so while it might be appropriate to sell to like a western governments there's maybe some more responsible use of surveillance than Saudi Arabia and I think the key distinguishing factor there is there's a couple of distinguishing factors one of them is oversight so to what extent are the people who are operating these spy tools accountable to someone and then to what extent do people who are wrongly spied on have the ability to achieve some remedy for that wrongful surveillance like Saudi Arabia where there's really no rule of law there's really no oversight if MBS decides he wants to spy on someone then he calls up his intelligence guy and gets him to spy on them and nobody can say anything about that so I think that in terms of picking customers it would be great if these companies were forced to achieve some or abide by some sort of principles in terms of vetting customers I mean they like to say well we have external vetting of all of these deals there's some external committee composed of experts that decides whether we want to sell or not but I think there really needs to be some requirement or some way to say well look there was no reason this sale should have happened you should have known that the surveillance was going to be abused if you sold to government X or government Y and what do you think about that particular balancing act where do you draw your line on that question there's so much in security as a balance and that's where I would love to start seeing more global norms even if they're not formal which means that they they're not going to be fully there's no repercussions for not adhering to it but can at least establish the baseline we'll know countries that adhere to certain aspects of those norms are not going to be using spyware for instance against their own populations, against human rights those kind of areas whereas we also know we more than willing to not want to adhere to those and there it is going to be a balancing act I can also guarantee that there will be some democracies that are not going to want to have to adhere to those kind of norms because they're going to want to use it for some kinds of national security events that they want to be looking into so it is really hard I think in the case of spyware however and I think you nailed it as far as doing what seems to be the best case scenario and having greater trust in what individual actors whether they're government or organizations how they're going to be using it and and implementing guidelines on those areas and then ideally pushing forth and getting more so to a place where more individuals also can protect themselves against those kind of spyware and so that also the other side of the coin is empowering a lot of individuals also to be able to defend themselves against that when it does happen and having the tools and not making it so complex that they have to be a computer science technology to actually be able to protect themselves and so that's where I'd like to see additional information being shared in that area for strengthening defenses against those kind of spyware because they're going to be misused at the end of the day we should take it as a given that whatever those tools are going to be some actor out there will misuse them You anticipated my next question I think that we've been talking a lot about government access and we should in government use of tools and governments using them badly or appropriately How do you translate those thoughts into I'm not going to say more benign but smaller scale uses of these tools and what I'm thinking of is kind of stalker spyware that can either be sold to figure out whether you're middle school or made at home or figure out where you're a strange spouse is Do we have to think about that differently? Can we think about it kind of in the same vein? I don't know who to point this at but I think Bill Okay so yeah I mean stalkerware as it's called or spouseware as it's also called this is sort of a very kind of creepy thing to think about I mean I think a lot of companies have been doing some interesting stuff here so want to buy occasional collaborators like Eva Galperin at EFF has been doing a lot in terms of trying to get companies at a bare minimum level to try to get antivirus companies to detect and warn you if this is installed on your phone or on your computer because the idea is these companies might say oh well there's legitimate uses here like let's say I have two phones and I want to use one phone to monitor the other phone or something so okay maybe we shouldn't have antivirus software automatically delete this but at least warning or prompting the user to make sure that this is not being done without their consent which I think is the key thing behind the use of this stuff like if the person who you're using it on is not consenting then there needs to be maybe you're a government and you've got a wiretap subject to appropriate judicial approval and oversight okay but other than those circumstances it's sort of hard to see how spying on someone else without their consent is okay and with monitoring children I don't know that sort of gets into kind of a tricky area I could definitely see wanting to find out for instance the location of your child if they're a minor where are they getting into trouble or something and I don't know that's sort of a trickier question which I I don't know if we had good answers no one would want to listen to us this is all tricky and this is all like very hard very hard questions you talked a little bit about actually I think all of you collaboration and the role that kind of outside thoughts plays and I'm thinking in particular you mentioned Eva at EFF another organization that I know at least I have worked a lot with is National Network to End Domestic Violence which looks a lot at this from that stalkerware perspective hence the question and I know that Facebook has a program where they bring in folks like NNEDV and others to kind of give an outside perspective do you want to talk just a little bit about the importance of that yeah so unfortunately I'm probably not the best person at Facebook to talk about this because I focus very squarely on law enforcement access and data protection issues we do take these kinds of issues really seriously and the safety of all of our users is one of our top probably our actual top priority and so we do have you know global safety summits where we bring in different stakeholders to really make sure that we're hearing everyone's perspectives as we develop our policies around public safety but the best person to sort of like talk about how those summits work and how that you know policy making process works probably are the people who actually do all right wasn't sure how involved they were in the kind of law enforcement side of things so Andrea do you have thoughts on collaboration and building these global norms that you were talking about yeah and I think one of the comments that I took away from the last panel was the criminals are doing a better job collaborating than we are as defenders and I think that's great hearing that coming from law enforcement because it's what I always think that and it's nice hearing someone else validate your hunches so across so many different areas we could collaborate better there's so much innovation that can still be had on the defensive side that I think we're just really starting to do it because we've had the pendulum swung so far towards locking down the data and that we're now not able to reap all those benefits of actually unlocking that data through collaboration whether it's through you know threat intelligence sharing to get better pictures of the threat landscape for you know for banks for fraud or what not or just for the IC sharing data in that area but doing it securely enabling that kind of collaboration to happen there's so much room to improve in that area where at that intersection of technology and people where that could actually happen and so even if we had the technology which does it you know existence increasingly there to help secure it and enable some of that collaboration we now have to get over some of those social and cultural hurdles of well you know it's my data and I'm not going to share any of it with you and so if we can do more of that data sharing we can really address a lot more of these threats a lot which then gets into some of the norms area where you know just sharing some of the lessons learned doing a bit more of the collaboration you know across the globe and you just recently example for myself you know I've been working a bit with a woman in Australia and just the different perspectives that I think we each bring to some of our side conversations is phenomenal and get like the encryption wall that passed in Australia hearing it from someone in Australia about how it happened and in that whole situation you know we can we can read about it here and read a fair amount about it but it doesn't replace actually hearing like what you know some some of the more finding details about society and what went through the legal process for how it actually happened and by having those discussions we can ideally help prevent somewhere laws from happening within the United States because again at some point we all get there's going to be another you know encryption battle going on and there's some side ones already but there'll be another FBI versus Apple or something along those lines that are prepared to help ensure that encryption stays strengthened so they're just there's just I think so many different areas where collaboration can help us both on the security and privacy side but this is another area as far as just innovation at large from doing joint research and all those kind of areas that we really have just gone so far back to just walking down all the data which is understandable and it's human nature we don't want to have all of our data thrown out there for all its legal repercussions for reputational for financial risk all those reasons and we need to get back to a bit more of a balance through some of the various forms of collaboration securely though I think that that's actually getting back to the this tension and all of this being a balancing act do we emphasize collaboration do we emphasize security are they mutually exclusive I hope not but I do think that kind of how do we create information sharing norms that are appropriate because when a lot of people say information sharing they they mean tell law enforcement what they want to know but that's not actually going to and this is the moderators injecting yourself you know that doesn't necessarily fix anything so I notice that and that's why we are seeing it within industries themselves much more activity within the information sharing groups and so the organizations are starting to take it on themselves again just for within their own industry to help figure out how they can collaborate and share more because they're getting attacked by the same kind of groups of people the various actors and so there are a variety of solutions the tensions are there but people are starting to push back now organizations are and I think governments are against what has been the status quo because that status quo is not sustainable Gotcha so in that realm you talked a little bit about interoperability and of Facebook's various messaging platforms do you see kind of opening that up as a place where you're going to be able to kind of share lessons across your various platforms at Facebook because I know that they have not always been interoperable working at a tech company I understand why it's hard to make these things work together sometimes do you think that opens new avenues for hackers law enforcement or actually makes it easier to defend so I mean I think whether or not it's easy to defend is probably more of a question for a technologist I think that the thing that we think about when we are making these decisions and really what animated our decision in order to move toward interoperability and tend to encryption across our messaging services was the fact that we do see cybersecurity threats increasing across the board we see threats to users data from authoritarian regimes increasing and we see just general threats to privacy increasing and a desire amongst internet users for more private means of communication and so we are trying to respond to that and make sure that we can provide users with those secure means of communication but also make sure that they're not sort of in a closed garden where you can only use one service and not another and if your friends or your family are using a different service then you're not able to communicate with them securely and so that is why we are creating interoperability between our different messaging platforms and I think you know one of the things that we find challenging is we do see increasing demands from foreign governments for technical assistance and so this is in different forms but the most concerning form is where governments are asking for ways around encryption or encryption back doors as they're colloquially referred to and this is something that is just deeply concerning because it really it's it goes against the principle of providing secure communications but in addition to that we as a company just don't feel that governments should be able to tell us we should be building our systems where we should be storing our data and we think that it's important to provide excellent security to our users' communications but that's also one of the reasons why you know we are really looking for ways to change how we respond to cross-border data requests because I think as encryption use increases as data is sort of like de-territorialized in terms of where it's stored questions about jurisdiction become really pressing and what we find is we have a mutual legal assistance process in the U.S. that for Facebook applies to the whole world except for the United States which works through its own judicial process which is very similar to the MLAP process or the EU which works pursuant Irish jurisdiction but the mutual legal assistance process requires that foreign governments that are seeking users' contents get warrants based on probable cause which is a pretty tricky thing if you're a foreign government and you don't have concepts of probable cause your legal regimes are different your criminal laws are different and so as a result of the complexity and the labor intensiveness of this process it takes varying amounts of time in order to respond to MLAP requests but on average when it all shakes out it can take around a year to respond to these requests so this means that a foreign government that is responding or investigating a murderer or another serious crime that was committed in their country by one of its own citizens could have to wait around a year in order to see information that could be relevant to that investigation we think that the amount of time that it takes and the difficulty in responding to these requests that the government has to be clear that requests as soon as we get the warrant pursuant to the MLAP request we respond immediately but the process, the intergovernmental process takes a very long time and that length of time winds up undermining public safety because it can impede legitimate investigations into serious crimes so why we want to address this one by supporting increased resources and expertise at DOJ and throughout governments around the world to help speed up this process but the reality is like all of the resources in the world and expertise in the world probably aren't going to address the problem wholesale and so we also support alternative mechanisms for accessing data such as the U.S. Cloud Act which passed last last spring and that would basically enable foreign governments that meet human rights standards to enter into executive agreements with the U.S. government and upon entering into that agreement they could directly ask providers for users' contents and providers would be able to voluntarily respond so we would still be able to apply our test to make sure that we consider the request to be lawful to be within the jurisdiction of the requesting government pursuant to their laws appropriate appropriately scoped and purposed and then also in compliance with international human rights norms if those elements weren't met we would be able to push back and basically redirect the request back through the MLOT process but under the Cloud Act this at least creates a supplemental process for human rights respecting governments to be able to get information more quickly so we'd like to see some you know more investments along those lines we haven't actually seen a government enter into an executive agreement yet so the negotiations for that are still ongoing and lots are slow and so are governments so the panel description kind of promises that we'll talk about proactive strategies to move us in the right direction specifically to protect user rights and to promote safe and smart use of these tools whether you're talking about the internet your kid monitoring software your you know any of these tools that have the ability to be abused so this is for each of you what strategies would you like to see focused on that would really kind of impact people rather than necessarily governments I will start I'll do a sort of multi-pronged one focusing on the tech just to look at what some of the basic steps are while there's no you I think best practice is the term that people in the InfoSec community hate but at the end of the day try and encryption as best you can and do be wary of any of those kind of track whomever tools for the potential for misuse those are some of the big steps and obviously phishing is still a big one but those are some of the steps as far as just securing yourself and just taking back greater control and we're seeing public opinion pull after another in the United States for people wanting to take back better control of their data and so get better informed in those areas and try to do that at the same time get involved with the discussions that are going on at the federal and in your state level as far as data privacy laws that are coming up every state more or less is now debating some of them and so I tend to focus on the tech community and try and get more tech people involved in shaping it but obviously the legal field absolutely especially people have legal and tech background are super important helping shape these discussions so they don't turn the wrong way and so they don't infringe on human rights and privacy or they don't go the other way and basically do nothing by doing having a really low bar for that so I'd get involved in those and there's a lot of different efforts that are out there that you can get involved and to help get more informed on what's going on within the United States at that level Yeah so at least from my perspective the way to what you want to do to help people who might be potentially victims of the abuse of these tools is to reduce the probability that these tools will be abused so you in some sense I gave an overview of what's been tried so far but I think where we've had less success is in holding the companies directly to account so let's say Jamal Khashoggi's friend Omar Abdulaziz in Canada was spied on by apparently the Saudi Government using NSO Group spyware wouldn't it be great if there were a way to hold the company to account such that this could dissuade the company from making those sorts of sales in the future or from continuing to support or maintain Saudi Arabia as a client because you know this sort of relationship between the companies and the governments is not simply transactional it's not like oh you give me money I give you spyware there's an ongoing relationship where there's support, updates, training those sorts of things so the tricky part though about holding these companies to account is that it's really really hard to gather evidence in this space like in order for you to get conclusive evidence that someone was targeted by some spyware well okay maybe you have to inspect their phone you've got to get in touch with the right person and then the question is okay well maybe they were targeted you can show that they received the dodgy email or the dodgy text containing the link that you can show is to the spyware but was their device infected was anything successfully privacy breached in some sense so that's sort of like a higher level to show and it's much harder because like well okay these devices we're carrying around nowadays are pretty much black boxes to investigators so Apple, Google they really really lock down the device you can't even get like if you want to do a forensic analysis on it it's pretty much not possible unless you yourself hack the device in order to get enhanced access do the analysis so you know thinking about ways to make it easier for people to access their own data and you know help with these investigations you know in that same vein there's also well okay you know if someone was infected with spyware right then their computer their phone whatever is infected is sending information back to some website some server and we know in a lot of countries there are data retention requirements where the ISP has to give your web history for like a year or something so that law enforcement if there is a criminal investigation can then go and get that web history and investigate but it would be great to have if we're going to have those data retention laws it would be great to have you know more control so that potential targets or victims of these sorts of attacks could say oh well I want to find out if I was targeted hey ISP can you send me my history my web history then I'll go to you know some research and take it to them and see what they can tell me or you know hey Facebook, hey Google, hey Twitter you know you sent me this really interesting state sponsored alert can you tell me perhaps what what triggered it can you point me in the right direction so that we can perhaps document evidence of this so I think there's two parts there's getting better evidence which involves putting people more in control of their own data and there's also the you know somehow perhaps creating some new liability for these companies that are found to be assisting in these human rights violations so yeah I think when we're thinking about how to best protect users and what are some of the new things that we can do from Facebook's perspective you know we are really looking at it in terms of what are the laws that are governing user privacy because when we're thinking about how to hand over user data we're thinking about it in terms of what are lawful requests when are we required to hand over data or when is there an imminent threat to life or serious bodily injury that would necessitate us handing over information and so what we're really thinking about is how can we make sure that the laws are more protective so that whatever government it is that's asking for user data it's doing so right's protecting way and so as a result of that we've joined the reform government surveillance coalition to work to amend US surveillance laws we're a part of a Digi coalition which is a coalition in Australia that works to address the assistance and access act which raises some real concerns around compelled decryption demands through technical capability notices which we were talking about a little bit earlier and so we really try to think about like who can we collaborate with in industry and civil society to make sure that laws are protective of people's rights so that governments can't overreach and so that there are appropriate oversight mechanisms in place so that if they do there's a means of redress through an independent oversight or judicial body great so my plan is to let you guys go a little bit early so I think we'll get started on Q&A I know that we're the furthest room from your closing keynote and I want you all to get the best seats so we would like to take a few questions I want to be clear they have to be questions I will cut you off and we will we will kind of if you can keep them short that's perfect like tweet length and because we want to get as many as we can in so I Bill I you talked about the spyware and I know that there are rules and so on but can this is a spyware good enough to be able to get into Facebook and I mean Apple I remember all this talk about Apple not giving out anything can some of that spyware get into my iPhone so the question just for the stream is about whether some of this spyware is good enough to get into an iPhone even though it's relatively hard to get into an iPhone if you're law enforcement can they get into Facebook which pieces of our digital life can this spyware get into well there are companies out there that are selling capabilities to get into pretty much the latest versions of iPhones of androids of any device that is you know commonly owned by consumers you know and once in the device they can access you know your Facebook data that way so I think that companies like Facebook like Google have a very sharp security teams that are focused on not only protecting employees of the company against these sorts of attacks but also users of the company's products so I think that the vector for getting into the phone would probably be a vulnerability that exists on the phone so like in WhatsApp WhatsApp was shipping code they unknowingly had a vulnerability in their code which you know when you got a call it could exploit some bug in the call signalling processing and take over the phone so pretty much they'll be going after the devices themselves and vulnerabilities that exist in the devices and it's a really hard problem because the devices we're carrying around in our pockets are running tens of millions of lines of code and how do you ensure that there are no bugs in it it's hard you know we've built up this multiple layers of software really really quickly without you know in many ways without serious thought about security of each layer so there's lots of bugs in our phones and companies are looking for ways to get in and selling that for millions of dollars this conference is trying to develop approaches rule of law approaches to these questions and going back to your suggestion about norms what would each of you identify as the top one or two or three sources of developing norms that those of us interested in these issues or building legal responses should look to I'd start with reviewing what the UNGG has already gone through and find out where some of the hurdles were because that will end up make basically there was some collective action problems so if you want to help avoid some of those I'd review some of those but I also would then I'd make sure to look at what some of the corporations are doing as well and even the Paris call oh yeah sure yeah absolutely sorry yeah so the UNGG basically did about I think about five different iterations trying to establish just common basically the low hanging fruit for what cyber norms could be and so it could be anything from not interfering in a cyber emergency incident response team so allowing governments to respond to those kind of incidents to not attacking critical infrastructure in peacetime some various areas along those the U.S. Sino agreement from 2015 was ostensibly to not allow commercial corporate theft by a government to help their own commercial enterprises that has since there's been numerous reports on that the trade rep U.S. trade rep did a really good report a little while ago outlining how that norm was broken but interestingly we can show how some of these things you spread when the U.S. China did that then the U.S. and Canada did something similar and China and Australia did something similar so while it was broken we started establishing some baselines for those kind of areas and that was all governments doing that the Paris call is an interesting one because that is an integration of both tech companies generally tech companies, corporations and governments and the numbers I actually don't remember it was over over 100 I think countries and then some like over 60 major corporations all signed on to that trying to rejuvenate some of these efforts towards norms again with some of those low hanging fruits and Macron introduced it in November the U.S. did not sign on to that for me I wish we would I wish we'd actually help shape it as well and that that's one of those things but by not being part of some of these discussions we can't shape them then we can't help to shape a lot of those discussions there's the Charter of Trust from Siemens is another one for corporate level that has gone out and then there's the Tech Accord that Microsoft put forth and Brad Smith from Microsoft I think it was RSA Talk like three years ago that kind of outlined some of those and if you hear the notion of a cyber Geneva convention it's linked back to that so those might be some good places to start but do you either Robin or Bill have thoughts on that one or should we move on to the next question okay we're good yeah so one of the questions I had was sort of you guys have talked about how collaboration between governments and between the actual companies themselves are creating the software is so important and then you also mentioned activists so like so there's lateral and then there's horizontal right so how can activists who maybe they don't know the citizen lab and they might be watching this streaming service right now how can they sort of get involved and let people know like hey like I think something might be up I need help I don't know who can help me because you know we deal with a lot of activists on the ground who really are dealing with serious threats and for that reason we oftentimes are like the connecting point to say okay well here's your problem here's your solution but what would you want sort of activists to know about like who can they go to who can they report this to who can they work within collaboration because a lot of the information you get is coming from these people and you but for those people you wouldn't even get the information so what would you want them to know maybe right yeah that's a good question I mean in terms of citizen lab ourselves we're a research organization so we don't really have a whole lot of capacity to be providing support to to activists around the world you know there are really great efforts like the creation of digital help lines like the access now digital help line which is sort of a first point where activists or civil society who are experiencing you know something suspicious going on can contact so they assist in things like oh my my account was hacked by somebody can you help me get access back to that you know or you know oh my my phone's acting strange you know what do I do because you know it's it's it's tricky because you know there are specific signs that that people can look for that should be red flags but but often it's not the ones that are sort of intuitive like we'll often get a lot of people saying oh well you know my phone's running really slow or my battery's draining really fast and you know it's sort of hard to know well okay maybe the phone's old so the battery doesn't last as long or maybe you just installed some app which is very very power hungry or something like this and slowing down your phone but you know being aware of the sort of vectors that you can detect like for instance suspicious text messages it is you know almost assuredly some sort of targeted attack if a political activist gets a message from an unknown number promising you know juicy activist details with a link or attachment in there right you know without any sort of context about who the person is from a legitimate institution or anything like that so so that is you know what ideally people should be looking out for but you know for as a first point of contact I think digital help lines like the access now digital help line are good for that sort of stuff yeah all the way in the back is great I haven't heard much about repression of First Amendment rights where speech rights outside of the United States and I know that Facebook has recently decided to cut the access of certain people most of whom the people in this room would find to be objectionable speech to begin with but some of us at least have some objection to repression of any speech at all and can we get some indication of how determinations are made as to who's cut out and who is not thanks so generally we will well not generally we will always apply our terms of service so when making determinations about whether someone is going to be suspended from the service it's a question of whether our terms of service have been violated and we have teams that work on issues around content moderation and the development of the terms of service who work to make sure that those terms of service and the standards that they set forth are applied to everybody and so that's that's it okay cool and I saw I mean I think that all those terms of service are on the website that's right and I think that's just common how websites and services try to communicate that so I know that our add-on store has a similar set of rules that we recently enforced which was great fun yes thank you sometimes when I send an email I proof it and then after I send it it's been altered where it's been changed to make me look stupid like I don't know how to spell and I have malware bytes and I have a look out on the android 3c well somebody's it's probably NSA I was in the piece born in Lawn do you have a question? what can I do to stop this? well so that's a great question I mean I think there's definitely resources you can take advantage of now you know digital help line and mention sort of what's going on yeah I see two more and then we're going to cut off questions I think so thanks really appreciate the panel and I really appreciate that you guys had the conversation about Spouseware and Stockerware I work for one of the AV companies is trying to figure out how to be more helpful in this space so I'd be very curious I guess maybe this is probably a question to Bill what else should companies be doing to kind of help users better empower themselves to protect themselves we just recently announced a privacy alert largely due to the leadership of Ava and others but trying to figure out what else should the industry be doing at large to help in this space that's a great question well I mean you know so the AV industry is sort of one part of it I think there's also like Google sort of adopted their policy about you know not running ads for you know how to spy on your spouse and things like that so I think you know that I don't want to sort of step out of my bounds here I think you should talk to Ava a bit more for details on that but I'd say that you know in terms of AV companies I think it's great that you guys are are sort of taking the position about you know notifying people if there might be something operating without their consent like that that's a great question all the way in the back thank you actually just to follow on that I've noticed when you mentioned the other countries using surveillance technology is this an ITAR issue that you feel software developers in the United States needed to recognize I apologize if this was covered before but need to recognize early on and then also data destruction at the border oftentimes when activists come to America countries have their phone sees or information sees Facebook pages looked at what types of software developments do you develop to prevent that and also communicate that to other countries that might be using them yes so I'll answer your second question first about about border crossings this is sort of a very tricky thing to navigate because you know in many cases if you refuse to submit an inspection you can't cross the border right I mean the exception is of course if you're like a US citizen coming back to the US they have to let you in but if you're someone who is I don't know like a refugee or applying for asylum or a political dissident who needs to travel for whatever reason you're pretty much at the mercy of you know letting them inspect your devices the best defense against this that I can think of is to not have if you're worried about data being being inspected to not have it on the device not have it accessible from the device so you know there's things you can think about which again are sort of just things you can do maybe tech geniuses can do but need to be you know incorporated into perhaps user friendly apps and things like this like you know one thing you could do is say okay well I'm going to you know maybe I'll think about my computer I've got some sensitive data I want to encrypt this data in a way that I can't decrypt it but when I get to my destination maybe I share like I encrypt it with like a public private key thing and I share some little bit of information with someone at my destination who will only give it to me when I'm there in person not under duress right and then without that bit of information I can't access my data that requires a lot of trust obviously but something like this where you can think of making it impossible for the data to be seized or decrypted when you cross the border that might be a fruitful direction there and I believe your first question was about regulating development of software like arms or weapons or things like this so I think there's been a bit of hesitancy in the tech community about that sort of approach given the crypto approach the crypto wars of the 90s I think though that there does need to be like stronger export controls are part of this the tricky part is identifying what exactly you want to control and I think one interesting direction to look here is the sort of nature of companies that are operating in this space so you know what's been tried before is okay well let's look at tools the intrusion tools so we'll add this to a list of dual use products and then it'll be regulated but if you think of it's not just the tools like these companies finfisher hacking TMSO are selling these sort of integrated packages where you have tools you have support you have you know they're setting up infrastructure maybe they're even helping develop bait content to get people to click on links or open attachments and that sort of integrated service I think is something which maybe you could reach out and specifically try and control great thank you guys this was a great panel and I appreciate it was great fun thanks for inviting us all and I promised I would let you go a couple minutes early we are exactly two minutes early since I know fire is on so thank you very much this is very important and I didn't for a while the schedule suddenly I'm noticing when I do something like a PDF or something else the big green rectangle do this at all click here first that's entirely legitimate that is okay now there's so I don't have like all this I hate it we've had quite the busy day and I'm pretty sure most of you who I know I have have learned quite a bit from our presenters and our discussions I'm really happy to be introducing our final closing session of the day my name is Linda Beshai I am the director of research evaluation and learning at the American Bar Association rule of law initiative we have titled our afternoon session a conversation on the future of the cyber landscape not to be very ambitious or anything that's just a modest discussion topic and to help us with this discussion I can't think of anyone better than Mr. Glenn Gerstel who is joining us as the general counsel of the national security agency before that he practiced for 40 years in the private sector and he has taught at a couple of law schools at Georgetown included as well as New York and he was on the president's national infrastructure advisory council which reports on security threats to the country's infrastructure so I think he brings us a very nice breath of private and government experience and to help facilitate the conversation is our own Judge James Baker a member of the Rolly board and someone who really has been extremely helpful with my team and I as we thought about how to put together this conference and what it should contain and how we should tap into this incredibly complex topic Judge Baker in addition to being a member of the Rolly board is a professor at Syracuse University and he's the director of the Institute for National Security and Counterterrorism at Syracuse before that he was the chief judge on the US court of appeals for the armed forces and the legal advisor to the National Security Council so I think we'll get an excellent informed conversation. Thank you very much. Is this on? Yep, you're good. Well thank you Linda. I'd like to start by just noting that we're in the Frank Carlucci auditorium I don't know how many of you knew Frank Carlucci a few of you a wonderful public servant who served his country in the world as secretary of defense national security advisor ambassador to Portugal but also for most of his career as a career employee of the government doing his job every day with honor so it's sort of exciting for me to come here and realize we're in the Frank Carlucci room which is not to say I was avoiding the George Schultz room or one of the other rooms but it is an honor to be in this room and to be on this stage with Glenn Glenn is doing something that Michael Hayden taught the intelligence community to do which is to get out and about and explain what it is the US intelligence community does and why it does it and how it serves a greater good that's a scary thing to do if you come from the national security world so but it's an important thing to do and thank you for being willing to do it here's what I'm going to do it's supposed to be the facilitator which I'm a potted plant essentially but I'm going to ask a series of questions to in theory get the ball rolling I'm going to the warm up like a goalie takes slap shots from the blue line to warm up the warm up shots for Glenn are just a little bit of background about his career about the NSA general counsel's office about the role of the NSA I'm then going to ask about the global situation could he give us a threat brief in essence about what's going on out there thinking about the elections ahead random countries like Russia, China Iran and then I'm going to ask him about emerging technologies and how they're going to impact on national security I have in mind artificial intelligence quantum computing and I have topics like Huawei 5G and things like that in mind so those are easy questions and then I want to save at least a half our time for audience questions if that doesn't get as far enough along I'm going to then ask some questions about cyber process and persistent engagement fair enough so first the slap shots to warm you up and get your glove hand going tell us a little bit about your background and about your role at the NSA first thank you Jamie I appreciate that introduction and also giving me the warning of what you're going to cover so that gives me a little time to think about it first let me say how happy I am to be here at the ABA ROLI conference but the ROLI initiative is just a terrific one and very very important and the fact that you're choosing cyber security as the theme of the conference for this year is really critically important and I hope to do justice to the topic even though I know I'm standing between you and drinks at the end of the day so to your question the national security agency has twin missions the missions are easily stated but the rest of it is all complex but the twin missions are one foreign intelligence surveillance and I emphasize foreign and the second is cyber security those are the twin missions that are easily stated and the details get complicated because it's a technically complicated area the area of law is very complicated on both sides and the environment in which we operate is also complicated we can spend some time on that I'm the general counsel of the agency which is a non-political position I came to that by way of a career in the private sector I didn't come to this job with a particular national security I always say axe to grind so to speak I was knowledgeable a little bit about homeland security not particularly knowledgeable about national security there's a bit of a difference and certainly new to the classified world so the first year or two for me was all about being thrown into the deep end and trying to learn how to swim and it was a fascinating exercise I've been struck by a couple of things at NSA it's a gem for the federal government it has outstandingly brilliant people probably the largest collection of PhD mathematicians on the planet and a group of really patriotic Americans who are truly interested in keeping our country safe both in terms of the foreign intelligence and cybersecurity and also being very very scrupulous about the rule of law so when I came in as general counsel I was a little worried what's the environment in which I'm going to be coming in here and I was able to see that everybody from the director on down is very very focused on making sure we comply with not only the statutes and regulations and procedures that govern us but also most importantly the fourth amendment Glenn you came from private practice how did that prepare you for this role and how didn't it prepare you is that a normative path to being general counsel of NSA well I wouldn't suggest that I would necessarily emulate my example so let's start with that I found it on just on a personal level as I hinted at before a little difficult to make the transition not never having really served in government before I'd been a volunteer boards and commissions but it's not quite the same as being inside government it's certainly not the same in the national security sector where it's a classified world which I was not used to everything from literally two sets of computers and two telephones one desk, one classified, one unclassified and on a personal level for 40 years or so I used to come home to my wife and tell her who was also a lawyer and tell her all about goings on at my law firm and she knew everything about my partners and my clients and now when I came home after work she'd say what'd you do today honey and I said oh some stuff she said well with whom oh some people and that's about all I could do so that was an adjustment I was getting a little flip about it but it was a serious adjustment that's a good answer by the way so I'd stick with that one ok I'll stick with that one but I did find to your question that the role of being in the private sector really does help me in this job yes there's a lot to learn absolutely but the skills of being a lawyer negotiating, understanding, being good at analysis I was a transaction based lawyer so I was used to new matters arising all the time and learning to be a quick study so there's a lot of the skills of being a lawyer translated well into being the general counsel certainly the law firm management piece was the same but what I hadn't appreciated is the, I'll say political I don't mean Republican versus Democrat but sort of interpersonal skills knowing how to deal with the interagency process which is a political process and again I don't mean that in a partisan way but you're just dealing with sensitivities and equities and issues and knowing how to navigate that which is very much something that a lawyer in private practice in a transaction based practice trying to get parties to agree in a deal a lot of those skills were very, very relevant to being an inside government thank you so ends the easy question look I've actually got a head out right now turning to your proxy role now is the D&I could you give us a sense, a survey of the cyber threats out there nation state cyber threats non-state actor cyber threats and depending on how that goes I'll then ask you what's the thing that you're most nervous about fearful of in terms of cyber threats so interestingly enough for the past I'm probably going to get this wrong I would say about at least five years there's been a number one threat that has been announced at a public hearing held essentially every January sometimes early February before congress where the director of national intelligence the director of the CIA the director of the NSA and other parts of the intelligence community appear before congress to have an annual global threat hearing the number one topic has been cyber when asked to rank the threats the number one topic for the last several years has been cyber and that's pretty fascinating the other threats are the ones you'd expect yes of course North Korea presents great challenges Russia, China etc counterterrorism all the obvious ones that you already know about but not everyone would have put cyber at the very top and I think that's absolutely correct and this is the intelligence community this is the intelligence community's assessment this is the intelligence community's assessment the number one threat our country faces globally is the threat from malicious cyber actors and it's all the malicious cyber actors that you know ranging from yes of course teenage hackers but far more importantly nation states unfortunately the barriers to entry in the area of doing cyber mischief are pretty low so that's why you can't have the proverbial teenage hacker but at the serious level the United States really pours its resources into trying to do harm to the United States and our allies the results can be significant we've certainly seen that with actions by North Korea Iran China and Russia to name the four that are the most significant malevolent cyber actors and the damage that's been done has ranged from everything from credit card theft to identity information being stolen in the case of China arguably the world's greatest transfer of wealth has occurred through cyber means where China has stolen intellectual property through cyber means and taken away from US businesses and US owners excuse me, of intellectual property North Korea has introduced malevolent viruses throughout the the world through cyber propagation which have damaged computers same thing it was true as I said for Iran and Russia as we also know we can spend more time on this has been particularly active in influence campaigns and trying to influence at least the 2016 election and probably other elections as well you say probably does that mean you don't know or you're not going to say no I think we do know we certainly do know about 2016 probably just because I was referring to older elections I mean I don't think we've done the same level of research back into say year 2000 or something but for purposes of 2016 the intelligence community came out with a joint assessment which was quite thorough, quite detailed it's available online it's available in an unclassified form which lays out exactly what Russia did in the 2016 elections in terms of their influence operations which were very significant if you have any doubts about it you can look at the indictment also available online it's a public document by special counselor Muller issued in I want to say February of 17 but I may have the date wrong which spells out exactly what the Russian internet research agency did and you have Gandy Progosian who's one of the Russian oligarchs and how they set up a systematic program to have false personas to use social media to use stolen credit card information to dupe Americans quite an extensive thing the indictment is public and it's a fascinating page turner should we expect more of the same in 2020? I think that's inevitable we've already seen a great difference in elections between 2016 and 2018 in 2016 to be candid I think we, America were caught a little bit flat footed in terms of our response to what Russia was doing and we saw the results in terms of, as I said significant manipulation of social media we also saw the dumping of emails designed to embarrass candidates through WikiLeaks and the morning of after the election in 2018 looked a lot different from the morning after the election in 2016 after 18 I think the United States Government was able to get organized in a way that it hadn't been fully in 2016 and we were able to take a number of steps to blunt and thwart adversaries taking actions there so you didn't see throughout the 2018 election the same kind of allegations and complaints that you saw in 2016 and that was because of affirmative actions by the United States Government to make sure that our elections were safe we're certainly going to repeat that in 2020 but I think it's naive to think that not only Russia but perhaps other adversaries as well who will have studied this playbook decide to interfere in our elections Do you think deep fakes are coming to the next election? Sadly yes so deep fakes you may have heard the buzzword are false videos or false audio tapes made to look like the real thing shows a political candidate doing something inappropriate with someone or saying something completely inappropriate in the guise of a recorded phone call or a video tape you can imagine that being posted to social media assuming it's all false posted to social media so it's on Twitter Facebook YouTube videos whatever it'll establish some level of currency and credence because people believe what they see and what you hear it certainly sounded like candidate X candidate Y and the thing about it is due to technology and the reason they're called deep fakes is it's almost impossible to discern it from the truth so these videos will look thanks to artificial intelligence and other techniques these videos will look real and will be almost impossible to discern from the fakes so if you're a candidate and this is out there you can deny it and say that's a completely fake video will everyone believe that will some people think well you know I did see it with my own eyes and it gosh it really did look like Glen Gerstel in that video that's going to be a serious issue that's cherry the so I'll hold off on the what's the thing that you really worry about so we can keep the happy mood going these are all the easy questions right but that is a segue to emerging technologies and could you please explain quantum computing to us in one minute or less what was the next question yeah at least one of you understands that joke how are you at that do you want to try that well that was not it was a sort of fake serious question to see if it was taken well in an extremely general way obviously not the technology but quantum computing is a different type of computing than the kind of digital computing that we now have based on a completely different system of sub molecular movement that enables a vast number of computations to be made at the same time very very quickly far outstripping digital computer capacity to my knowledge no one's ever built a quantum computer it exists in theory people talk about it you see some literature about it but the theory again I emphasize theory is that if someone ever builds one of these things it will have the ability to perform all sorts of incomprehensible data crunching with the result that it will hopefully enable vast advances in everything from weather forecasting to disease analysis to national security perhaps breaking encryption so the Chinese have announced this is again all public they've announced it they say what their plans are so their plan for the year 2025 and beyond is to become the number one country in quantum computing so that's the challenge that's laid out for the rest of the world they've announced what their plans are and we'll see whether this ever goes anywhere but certainly has the potential for both good and evil in the national security sector you did very well unlike prime minister Trudeau who had the question planted I did not tell Glenn that I was going to ask him to explain quantum computing it is computing based on quantum physics which is the physics at the subatomic particle level which is chaos and it's taking these two concepts called superposition and entanglement and using them to create a new form of bit and on the basis of that if you can actually do it they predict they and they depends on who you're talking to and where you're talking it could be exponentially faster than a classical computer and some say as much as a million times faster but think about that now some of the limitations in computing today are just you can't do enough of the calculating and the time required but this would change all that if it happens so that's one technology artificial intelligence is a collection of other technologies I won't make you do synthetic biology but you can take your pick of technologies and talk about what's coming or you can switch and just say let's talk 5G instead because that's another form of technology so actually we can do that's an easy question so we can do them all which is to say if you hinted before what keeps me up at night so let me tackle that because it's the same point which is this is sort of my personal view it's not as an official NSA policy this is more of my personal view but being informed by my position so I'm most worried about a completely unprecedented and I would say almost incomprehensible level of change in the technological world ranging from the things you pointed out which is artificial intelligence 5G which is a completely new form of telecommunications that will be much faster like we talk about that for a second who knows possibly quantum computing we already have cloud computing just the whole onrush of technology what some have called the fourth digital revolution is if we're not already in it we're on the cusp of it it's already about to wash over us and I don't think our society has come to grips with this both in terms of our laws in terms of what privacy means in this context of this digital revolution where you're whereabouts and shopping habits and everything known so what does that really mean to privacy I don't think our society's broadly the western nations and indeed even the global norms haven't been established in this area with every other technology that's come along whether it's railroads, electricity take your pick we had a number of decades in which to sort out the basic rules of the road whether it should be publicly owned, privately owned how it should be regulated how it should be done how it should be done so we now have an electric system we understand the rules of the road you can't plug anything into it unless it's underwriter lab approved just to take one simple example but we have an internet and you can plug anything you want into the internet indeed by some estimates there are going to be over 20 billion devices connected to the internet perhaps next year and just to put that number in context that means when we wake up tomorrow morning there are 5 million more devices connected to the internet than today and when we wake up the next morning there's another 5 million and these are internet of things devices your telephone, I'm sorry your refrigerator your toaster, God knows what's going to be connected to the internet what are the privacy implications of that what would a nation state do if it wanted to do surveillance in that area what should the rules of the road be that there be certain base level cyber security in this and how are we going to deal with that all these questions are yet to be addressed and yet in 20 years from now or even 10 we're going to be dealing with this in a way that we have not yet organized our society to address these whether it's government institutions the rules, the norms, the laws etc so it's coming at us very quickly before we can adapt to it I'm going to turn to the audience now and ask if there are questions you'd like to put to Glenn and while you're forming up your question I'm going to give you so I will call on you and actually I'm used to calling on people as a judge so I will call on you if you don't call on us first and I know some of your names so stand by Alberto so here's a dealer's choice there ought to be a law and now fill in the blank or what is your position NSA's position or anybody's position that you'd like to identify regarding whether NSA cyber command should be single-hatted or dual-hatted so you get to choose which question you answer before we have the audience pipe up I'll take the latter because it's very discreet and clear so right now the director of the agency that I work at, the National Security Agency is General Paul Nakasone he's a four-star general, a full general he also has a second job which is he is the commander of the United States cyber command which is one of the, I think, 10 other so-called combatant commands in the United States military which are joint commands and that was a recent change in status so he wears two hats so to speak cybercom is, although I'm going to cartoon it a little bit this is not the exact way to phrase it it's more at what layman have said or others have described as sort of offensive operations this is the military using cyber to engage with our adversaries overseas and to try to deny or disrupt or deter them from malicious cyber activity aimed at the United States the same general Nakasone is also in charge of the National Security Agency which does not have that mission we're much more about finding information out about our adversaries as well as providing defense to national security systems not all of government just national security systems so where's twin hats each mission informs the other because obviously if you're good at cracking and hacking into foreign adversaries computers if we wanted to say break into al-Qaeda's computers you'd need to know how to do that which would be good to know on the offensive side because we'd like to know so there's a lot of synergies between the two some people have said it's too big a job for one person it should be chopped up into two others have said there's an inherent conflict between the two missions and it would make more sense if one person headed each there's been a series of back and forths on that no decision has been made my own personal guess is that at some point it would make sense to do this and that the government will do this but whether that's imminent or not I personally my personal view is I doubt and I think it's something that we need to see the two or the U.S. cybercom which is seven years old something like that mature a little more before there's a split but that's just my absolutely my pure personal view so Bill does General Nakasone have a view or does he defer to SACDF his I'm sure he does have a view but he's basically said that he's provided his military recommendation to the secretary it is the secretary's decision obviously with the president's concurrence but no decision has been made and as I said my own personal guess underscore personal this is probably something that's a few years off but it's ultimately the secretary's decision thank you okay let's hear some questions please up there in the back you have to do name rank and serial number and there'll be no speeches permitted okay General Counsel Goodell my name's Bob Allen I'm a recently retired naval intelligence officer and the question I had for you is today we were presented some great robust dialogue regarding fighting cyber crime by practitioners of law enforcement national security what I wanted to ask you is do we have our own legal self deterrent of using offensive cyber means against international organized crime because that's one of the few tool sets that was discussed today as we looked at fighting international organized crime or cyber crime specifically thank you if I thank you for your question and also your navy service if I understand your question what are the tools that we have to deal with cyber crime offensively using cyber against cyber crime international sure so there are lots of ways we deal with cyber crime and of course there's a multiplicity of actors involved in cyber crime as we alluded to before everything from individual criminals to nation states the United States government has been quite vigorous in going down the criminal prosecution route and I know you heard the Department of Justice earlier today the Department of Justice in conjunction with the FBI both on the criminal investigation and I think taking a pretty robust and vigorous response to cyber crime but I think it's important to understand that there's no one magic bullet there's no one size that fits all situations so in some cases in conjunction with the FBI both on the prosecution side and the investigation side has been vigorous about going after cyber criminals whether they're domestic or international including we've indicted a number of Chinese, North Korean, Iranians and Russians in this area have all been indicted and you say what's the point of inditing someone overseas well they can't travel number one makes it awfully hard for them to travel they know it in some cases we also combine it with sanctions and cyber mischief of 2016 so that could affect their ability to engage in economic matters and of course in one or two cases some of the people have been made the misstep of going to a country where there was an extradition treaty with the United States, the United States is extradited and they have indeed stood trial so certainly one is criminal prosecution the United States does have the authority and General Nakasone and others have the authority to engage with their adversaries in taking steps to counter cyber criminal activity so there are the requisite authorities it isn't always the case that a cyber attack has to necessarily be responded to with a cyber response the response could be diplomatic it could be through the criminal prosecution that I've mentioned it could be economic sanctions it could be a whole range of things sometimes in concert so it's possible that there could be actions taken that wouldn't be public so we wouldn't necessarily know that something's happened or something's been thwarted but that has so it's a wide range of things it is definitely something that is thought of carefully at the highest levels of government I've participated in some deliberations there it's something that's taken very seriously and I think the government a raid against a very very complex challenge is doing a good job but it is a very complex dynamic challenge I'm not minimizing the extent of the problem thank you right here Mary Greer and then we'll go over there I want to thank you both I'm Mary Greer I work at the rule of law initiative and I'm curious to know we have other implementers in the room but I think we also still have members of our donor community any ideas in terms of working with judicial actors prosecutors, judges defense bar civil society organizations any ideas for our donor community in terms of crafting programs and projects that might address some of the issues some of the cyber threat issues whether they're regional priorities that you see or you know against specific actors I wish we had an hour to respond to the question and have a dialogue on that but there's a wide range of things to summarize a few points so one I feel very strongly and I know General Nakasone has spoken about this too it's just going to be critically important as we face the future threat that I described before this onrush of technology and the problems of cyber security it's really going to be critically important that the private sector gets way more engaged and way more involved than it has right now and this is my personal view not an official NSA policy but there's a bit of divide there's government over here and the private sector over there the private sector says to the government gee you should be helping me and make sure that North Korea doesn't steal my credit cards or that China doesn't steal my intellectual property etc and it's pretty clear that if any one of those countries had god forbid dropped a bomb on us we know what happened to China as it appears to have done went to Marriott and stole several hundred million individual identities passports credit cards etc the United States government at least did not visibly respond with jets flying somewhere understandable prior to my prior comment but the private sector says gee we need we need more support in this area on the other hand the government is saying also the private sector has a lot of cutting edge they're the ones receiving these bearing the brunt of these cyber attacks and they're the ones who have more information they have more resources that are I think the net result of this is we need to have a new partnership a new arrangement between government and the private sector that would operate much more closely together in terms of addressing this and then secondly a massive amount needs to be done in education you could pick any one of these points whether it's if you talk about the effective influence operations one of the reasons influence operations make sense is because people don't have a real grounding in civics they don't really understand exactly what the government does what the rules are what the laws are and so they fall people can fall prey to misinformation so education on civics cyber security some of the issues we've been talking about is all important I think there's a whole panoply I guess the good news is there's a lot of action we can take in order to improve so if you want to be glasses half full person you can look at it that way could I invite make a suggestion you said you'd like to talk about that for an hour you're welcome to stay for the next hour but I know you have to go soon so instead could I suggest you have Roli the rule of law initiative that works extensively overseas doing things like civic education which is a great target of opportunity for some of these international partnerships we also have the president-elect of the American Bar Association here I'm sure that she would welcome any recommendations your team would like to suggest to the ABA side about how we can advance that kind of partnership with the private sector as well as the international private sector it's a pretty good target of opportunity for you all so I invite you, I don't know whether this will be welcome on their side but I invite you to task them to perform various things sounds good to me okay another question so we have this question and then we have this question and then we're probably riding to the end there I'm John Donovan student at the University of Connecticut School of Law and I was just wondering how closely you work with tech companies I mean I'm sort of thinking specifically here like social media companies and the Russian hack do you collaborate in terms of best practices do you point out problem areas that you see that they might not how does that work between you guys thank you sure good question the short answer is yes we do we the National Security Agency and the intelligence community generally I think has a good relationship with the tech community I think sometimes you'll see articles that make it sound like that's not really the case and yes there are different approaches on a couple of issues but my feeling is that the tech companies generally including social media companies they're patriotic they want to do it right they're worried about national security they understand some of the threats that intelligence community points out to them so I don't I think some of the some of the news media descriptions of a great divide between Silicon Valley on one hand and government on the other are a little overstated having said that it is definitely the case that there are differences in approach most notably around encryption and some other areas so yes there's areas that need to where we need to have more dialogue but broadly speaking NSA does enjoy a good relationship with with the tech community during the 2018 election and this was publicly reported the intelligence community worked with the FBI and the Department of Homeland Security to engage in social media and you read lots of reports about accounts being taken down and so on and so forth so there is a good dialogue is there room for improvement? Absolutely but there is an ongoing dialogue Thank you okay unfortunately this will be the last question just because Glenn has duties to country Glenn it's wonderful to have you doing the duties to countries that you're now doing you talked about the fourth digital revolution who are you? I'm Peter Bogger I'm a member of the rule of law board are there some legislative initiatives that could move quickly enough to either anticipate or to try to respond to some of these digital revolution problems that we have or for that matter treaties that we might be negotiating or starting to negotiate that ought to be priorities for us Thanks Peter the short answer is yes the longer answer is given I think and this is again my personal view not an NSA official view I think this is just an enormously complicated and challenging area which is going to make it difficult with either international treaties or legislation long term I do see that occurring just because I think the pressures for it and the recognition of the need for it are so significant so at some point I could envision and again my personal view at some point envision some kind of a treaty on international cyber norms we're not there yet we're far away we see nascent efforts we the judge talked about 5G so we see nascent efforts among allies in other countries to coordinate on an approach to dealing with the risks posed by Chinese equipment in the 5G area so there's some level of international coordination domestically there have been a whole series of bills ranging from ones dealing with supply chain issues to mandating that internet of things devices have minimum levels of security so there's lots of areas for the potential for legislation so I'm not saying that but my own personal view is this is such a complicated area and to go back to a point I made before this is coming at us so quickly that we don't have the time to sort out and think about try this try that analyze various options and come to a legislative solution that's just hard to deal with that does beg the question there ought to be a law assuming it could get passed do you want to suggest anything because we have this energy that we want to use and use in a productive way but if we all scurry about trying to do different things you are the opportunity to try and prioritize our energy what would you focus on in response to this question treaty bilateral treaty partnership statute executive order that does X speaking only for yourself even with that caveat I'm a little hesitant to me I think the area that needs the most urgent attention is the and maybe it's not a law but maybe it is or maybe it isn't is the one where we can try to get the private sector to get more involved in the area of cyber security and assisting the government and also for the government to equally be involved in assisting the private sector I think we need to get the marriage a little tighter and a little we need to take advantage of the strengths of both sectors and not have as much of a divide as I think we do that's just my view it's not necessarily a law but I think that's where our society needs to go well thank you that's a very honest answer and I appreciate that you gave it that you gave the official NSA and DOD position there you know thank you very much Glenn thank you for joining us if we could now thank our guest Glenn for coming down Linda has a couple of comments if you could just hold places for one moment just a few closing words thanks everyone and I really do appreciate that our conversationalists kept the conversation at a level where people who went to law school because they didn't do math like me could understand and appreciate that was very thought-provoking I just want to say thank you to a few people who really need to be thanked publicly because without them this whole event couldn't have happened starting with our USIP partners who were a pleasure to work with in particular Chelsea Dreher and also Philippe Le Roux-Martin and Jonas Klaas they were extremely helpful I also I also want to mention the names of our Roli crew because it certainly was a group of people our outreach team Lorraine Cook Hope Ann Roberts David Deppman and Jeremy Purcell extremely useful and there was those of you who were panelists know this but most of you don't this is actually an issue conference working group at Roli and we all put our heads together to flesh out exactly what this conference should look like and to make the invitations to think about who the speakers should be and how these panels should be composed and fleshed out and that's a lot of work and that is why this conference was so interesting and the working group members were Mary Greer, Tara Mubaraki, Samantha Tew, Shea Wilcox, Susan Goldman, Jesse and their work is great and there were of course on the day today many volunteers taking notes manning the registration table and the microphones etc. I just want to note to you that we are going to compile the notes taken today into a conference report that will be posted online and you should be able to see that along with the videos that were made today and the soft copy of the conference paper that website is going to be mbar.org slash cyberspaceirl So with that I would like to thank our speakers again and let you know that the reception is right outside.