 Ladies and gentlemen, welcome to New America. My name is Harvey, wrangled panelist, if I may. My name is Ian Wallace. I am the co-director of the cybersecurity initiative here at New America. Often I find myself in the position of moderating these discussions. Today I'm very happy to be able to offload that responsibility, so I am one of the panelists. But I just quickly want to take this opportunity to welcome members of the Pitt Center up at the University of Pittsburgh and the American Bar Association. One of the things that we try and do with our cybersecurity initiative is provide a platform for other people who are doing good work. Our funding comes from the Hewlett Foundation and from new partners at Florida International University. It gives us the opportunity to use this space to get good ideas out into the mix. So I'm going to quickly hand over to our good friend and member of the Pitt Center crew, Kirsten Todd. She's going to introduce our keynote and then moderate our discussion. Kirsten, oh dear. Thanks very much, Ian. And thanks very much to New America and to the American Bar Association for helping us host this event. As Ian said, so this is the event of the University of Pittsburgh Institute for Cyber Law Policy and Security. It's an institute that was started in early 2017 to provide a unique interdisciplinary environment for tackling cyber challenges. The objective of the institute is to really look at the intersection of law and technology and innovation. And one of the advantages that we're really proud of, and we have some representation of our academic strength here, is bringing the breadth of one of the world's leading public research universities to address these critical questions and to look at the issues around networks, data, law, policy, algorithms, with a focus on these changing gaps that are occurring in policy law and technology. And the institute's legal policy and technical researchers are engaging with policy makers in industry to create both the actionable proposals and solutions as well as identifying the issues that, as we like to say, understand the future as it arrives. So it's really understanding where are we coming from? Where are we right now and where do we need to go? And I think particularly in an era where we talk about cyber moonshots, whether or not we actually agree with that term, but we're always trying to anticipate what comes next. Being able to bring this interdisciplinary approach from one of the world's leading research institutions together, particularly to tackle the issue of cyber security is unique and really is exciting on so many different levels. And I'm pleased, Paul Cohen, whose Dean will talk more about what he's doing. His focus is really looking at this interdisciplinary approach in the school. And I think when you look at what the University of Pittsburgh is doing now and what it's seeking to achieve, being able to pull these disciplines from so many different angles to address arguably the most complex threat right now to our national and economic security is very valuable. So it's a great honor for me to be able to introduce the founding director of the institute, David Hickton. David is the founding director of our institute and prior to coming to Pitt, he served as the United States Attorney for the Western District of Pennsylvania. He was nominated by former president of Barack Obama in May of 2010 and was confirmed by the Senate later that year. Prior to becoming US attorney, David was in the private law practice specifically focusing in areas of transportation, litigation, commercial and white collar crime. In 2016, Pitt School of Law named David one of the distinguished alumni in that same year, the legal intelligentser named him Pennsylvania attorney of the year. Pitt recognized David as a legacy laureate in 2013 and also that year presented him with its 225th anniversary medallion, an honor bestowed on an alumni who has brought particular honor to the university through his work and service. During his tenure as US attorney, and this is where it gets very interesting and most relevant, he had several signature achievements and I'm not gonna take the time to walk through all of them but I will mention three that I think are particularly interesting both because of current events as well as because of why we're here. David spent a lot of time and really blazed a trail in combating the national heroin and opioid crisis. And as somebody who worked on these issues back in the early 90s when nobody was talking about them, the fact that he created so many efforts that are now becoming the gold standard for how to look at this issue when we are dealing with one of the greatest health crises of our nation is pretty remarkable. He formed the US attorney's working group on drug overdoses and addiction which includes key federal, state and local law enforcement along with public health and safety staff. He brought community impact prosecutions against large scale heroin traffickers and also worked to reduce the stigma around addiction. He also created several efforts to enhance community police trust including assembling a community police working group in 2011 to build trust for and from law enforcement and the citizens they served and he established a civil rights section in the US attorney's office. If you ask David what his passions are it's around civil rights and what happens with people. And I think you can see that both in the work that he's done as a lawyer and who he is as a person. And most importantly for our purposes of today he focused significantly on addressing the cyber threat. He brought the first of its kind indictment against five members of the Chinese military for economic espionage against Pittsburgh based companies and organizations. He created a dedicated section in the US attorney's office to focus on cyber crime and national security. And lastly but certainly most importantly he prosecuted the groundbreaking cases such as dark code, the largest English speaking cyber crime forum and Bogachev the creator of Game Over Zeus and Crypto Locker malware among others. As you can appreciate having David at the Helman as the founding director of this institute says a lot for not only what we're looking to achieve but what we will likely achieve. And it is with great honor that I introduce David Hickton today and look forward to the discussions. Thank you Kirsten for that warm and generous introduction and more importantly for your role as our resident scholar here in Washington DC. If you do not know I'm surrounded by rock stars at the University of Pittsburgh and Kirsten's one of them. She's had a great legacy in this area and our chancellor Pat Gallagher at the University of Pittsburgh was the former head of NIST and helped develop the criteria for cybersecurity resiliency with Kirsten's help and served on President Obama's cyber commission. 12 people selected across the whole country to do the look forward on cyber and Kirsten was the executive director of that. So we have some real cyber horse power. You're gonna meet Paul Cohen later today. We're very proud of what we're doing in this space at the University of Pittsburgh. I wanna thank the American Bar Association Cyber Scoop and New America for hosting this great event and thank all of you for attending. I really do believe that the cyber digital challenge is the challenge of our age and at its core both in my time as U.S. attorney and now as head of Pitt Cyber, I believe the central challenge is how do we apply law to digital space and how do we narrow the gap between law and technology. When I was United States attorney and I was doing outreach, I frequently said to people that I wanted to invite you into this cyber battle with us as a cyber warrior but here's the challenge. We are dealing with clever adversaries who are hiding in the shadows and the evidence that we are looking to identify evaporates unlike the prior era where it was static, it evaporates and we are dealing in a technology which changes so rapidly that we can barely keep up where the legal platform is effectively the stagecoach platform of yesteryear. Just think about the phones that you have had in the last five years and how you mock people who have a flip phone today and the applications that you do on those phones and you see that Moore's law has effectively been turned on its head in our own experience but when you talk about the legal platform we deal with and I'm gonna tell you some examples from the cases that Kirsten mentioned, the law has simply not caught up. The two principal challenges I faced as US attorney when trying to do cyber law enforcement were dealing with Mutual Legal Assistance Treaties or MLATs which are the principal vehicle we work with our allies, not our adversaries to share information to deal with this borderless crime and this international challenge of capturing cyber criminals and the issue of extradition. I mean, imagine what I was thinking on May 18th, 2014, as I was driving here from Pittsburgh to announce with Eric Holder the indictment of five members of the Chinese Army whose pictures we were gonna put in front of the world. Think of all the questions that were gonna arise as a result of that bold act and think of it in the context of be knowing that we were not gonna be able to extradite them. What did that mean? At the same time, what was at the root of that effort was leveling the playing field for American citizens because what was occurring and demonstrated by that case for at least a decade was that our American companies funded by capital provided basically by American stockholders or citizens like you were making an investment, deferring income, making a bet that we could find the new products and the new processes of tomorrow by deferring income today because we are still the cradle of education, we are still the cradle of innovation and the cradle of technology and our edge in an increasingly world economy is that that bet we make will pay off because China is at least seven times our size and most reports acknowledge that China will become the number one economy in the world by 2021 but it is still here where everyone wants to be educated and it is still here that we recognize through intellectual property and copyright protection that innovation is encouraged and we make that bet based upon the fact that we can harvest a return. So when I came to the office of U.S. Attorney in Pittsburgh in 2010, one of the first meetings I had was with the chairman of U.S. Steel and the head of the United Steel Workers, two iconic institutions in the country and in Pittsburgh and I was trying to appeal to them to help me do a mentor program for children at risk in our disadvantaged communities and they agreed immediately but then told me that what I really needed to focus on was the drip drip drip theft of our assets and they explained to me that since at least 2004 they knew they were being hacked but they didn't have any data on it. Frequently they would tell law enforcement and law enforcement would work on it but not share with them or law enforcement would come in and tell them they had been hacked and they would cooperate by allowing a box or a capture device to be put on their system to harvest what was happening and then never be told what that harvest was. But what they knew was is that products that were representing their innovative efforts and their investments were coming back into the market in a copycat manner and they were being supplied by state-owned entities overseas and produced at costs which were less than it cost us to make them. And what we knew was that China had entered the World Trade Organization in 2001 and in a decade by 2011 we had lost 50% of our manufacturing jobs and they wanted action on this and so we had to change the metrics for how we did this. At its core that problem was as old as any cyber case and as new as any cyber case. It's about attribution and it's about defeating anonymizing efforts taken by your adversaries. But the will to bring that case came from the past administration. The initiative came from Western Pennsylvania and we were able to get a case over the finish line that no one believed we could bring. But the legal challenges and the legal questions just proliferated. What if China arrested our army? Who owned the internet? What did extradition mean after 2014? What were the limits of privacy? What did this mean for our American multinational companies who are operating in China? And how are we gonna apply these questions going forward based on consensus and otherwise? The case against China received much acclaim and much criticism. I'm happy to tell you that most people now recognize that when President Xi came here in September 2015 and went to the White House and made an agreement with President Obama that is enforced by an annual meeting between the Attorney General and the Department of Homeland Security Secretary and their counterparts to verify this agreement that we changed the course of the conduct of our principal economic adversary today and going forward. But we didn't bring them here. They're not on trial in Pittsburgh. And what does that all mean for the future? In the Bogachev case, which happened just two weeks later in June of 2014, we invited the person who is generally accepted as the most notorious cyber criminal in the world. Bogachev had vexed law enforcement since 2007 with his Zeus malware, which was basically a crude botnet that came in by way of spearfishing and harvested people like you's financial credentials and when you weren't looking, moved money out of your accounts in great numbers. By the time we indicted game over Zeus, it was Zeus 3.0, he had adapted his system to avoid law enforcement capture from a centralized command and control to appear to peer system. And that meant that when we attacked it, we could only neutralize it, but we couldn't effectively destroy it. We put together a renewed effort to deal with that. We put a civil tool in the toolbox where we got a court order out of Pittsburgh to seize the domain. We worked with DHS to sink all the communications from the bad guys. We created a patch for the victims and it is generally accepted that we stopped game over Zeus and crypto locker in June of 2014. But he has not been captured, notwithstanding I was brought back down here to offer a $3 million reward on him. If you pay attention to the news when President Obama announced the election sanctions in December 16, they used the wanted poster of Bogachev to depict what a Russian cyber criminal looks like because the signature of anonymizing efforts by a Russian endeavor is to subcontract through a cadre of criminals, whereas the signature that China uses to avoid capture is to create hop points across the country to anonymize the IP address when they come in from Shanghai. Bogachev is still on the run. We had 12 countries help us with Bogachev. We went solo against China, but that reflected progress and the effort to apply a lot of digital space was gaining steam. It was establishing some norms. Those norms were enhanced by President Xi saying, we're gonna still spy on you, but we're not gonna steal your intellectual property. That was progress. Having 12 countries on Bogachev was progress. Having 20 countries on dark code became even more progress, but the question in Bogachev was as much about the mutual legal assistance treaties as it was the extradition. We have great relations with the Five Eyes and we have these treaties, but let me tell you, it's like watching a bad tennis match when you try to send an application over to England and say, I wanna get some process to secure this information. A neutral judge over there will deal with it. You have to negotiate with the chief legal, the opt-at who's assigned there from the Department of Justice. They come back to you with what they think is achievable. You present it, it comes back to you. Then you have to present it to an American court. An American court gets to look at it, then you present it back and you can just imagine in the digital age how unworkable this is. So one of the great things we can all do going forward is stand for improvements in that arena. I don't need to tell you that the stakes are high here. I think you understand that by your attendance, but whether we're talking about fake news, election integrity, a tax on our critical infrastructure, a tax on our financial system, or a tax on our energy and our energy independence, we have a great stake in solving these problems. When we announced the case against China, Director Comey of the FBI went on TV and said there are two types of entities in this country. Those have been hacked and those who think they have been not. People ask me every day, including in a program yesterday at Pitt, have I been hacked? And I'm incredulous. You have all been hacked. And we have not yet developed the proper response. So the questions going forward are, what is private? What are the bounds of international law enforcement? What are the rights and responsibilities of victims? What are we gonna say about reporting? And is that universal across the country? Are we gonna allow victims to fight back? And what are the boundaries of that? Are we gonna allow cyber bombs to be lodged by private entities that against hackers? Are we gonna confine our norms too? We're gonna help increase the cost of hacking by opening up the court system to greater access like Microsoft just did in the Eastern District of Virginia by re-seizing the domains of the Russian hackers, getting a default judgment, and getting an order which would allow a court to impose contempt and seizure of Russian assets. That was a watershed case and a watershed event. So we have a great panel for you here today. I'm gonna come back up and participate in the questions and answers. It's an honor for me to be here. This is a very distinguished panel. I think this is the beginning and continuation of a conversation that we invite you all to be on board. Thank you very much. Thank you very much. Very pleased to moderate this distinguished panel and I will spend some time introducing each of them. Although each of them said, you can just give me one sentence about myself, but I think that would be a shame for all of you because of their tremendous accomplishments. I'll take a few minutes and then we'll get into questions and then as David mentioned at the end, we'll do Q and A with everybody. So to my left is Paul Cohen. Paul is the founding dean of Pitt's New School of Computing and Information and that was just launched this year. And I wanna share two quotes from Paul that I think really represent what he's looking to do in this school. The first is from his welcome letter. He states, we recognize that Polly Matthew, and for those of you who don't know what that is, the ability to work in multiple disciplines depends not on mastering multiple curricula, but on learning what disciplines have in common. And he explains, I wanna promote systems-oriented research, technology and education at Pitt because the world's systems are increasingly stressed and we need new methods to model and manage them. And I think if there is any way to really look at what we're addressing in cybersecurity from that systems approach and from multiple disciplines, these two sentences say it quite well. Prior to becoming dean, Paul was the program manager at DARPA and he joined DARPA from the University of Arizona where he was a professor and founding director of the University School of Information Science, Technology and Arts. He also served as the head of the university's Department of Computer Science. And prior to joining U of A, Dr. Cohen worked at the University of Southern California's Information Sciences Institute. At that institution, he served as director of the Center for Research on Unexpected Events and deputy director of the Intelligence Systems Division. He began his career as professor of Computer Science at the University of Massachusetts and he's been published extensively. You can appreciate why somebody like him is exactly right to start this school and what it will be driving. To his left is Harvey Rishikov. Harvey is director and convening authority for the Office of Military Commission and he was formerly a senior counsel in Crowell and Moorings Privacy and Cybersecurity and Government Contracts Groups in DC. Prior to joining that firm, he was dean of the faculty at the National Defense University. He is the chair of the American Bar Association Advisory Standing Committee on Law and National Security. He's the former co-chair of the American Law Institute and is a member of the Council on Foreign Relations. He has held multiple positions in government focused on national and cybersecurity issues, including as senior policy advisor to the National Counterintelligence Executive. He also served at the FBI as a legal counsel to the deputy director of the FBI focusing on national security and terrorism and served as liaison to the Office of the Attorney General of DOJ. And then in a turn of positions to his left is Ian Wallace who is typically at a podium or usually doing what I'm doing but it's great to have him on the panel. Ian is the co-director of the New America Cybersecurity Initiative and a senior fellow in the International Security Program. His research is mainly focused on international security and the military dimensions of cybersecurity. He's also a member of the Future of War project. Ian joined New America from the Brookings Institution where he spent two years in the foreign policy program as a visiting fellow for cybersecurity. And he was previously a senior official at the British Ministry of Defense and before that he was at the British Embassy in Washington as the Defense Policy and Nuclear Counselor. Before joining the embassy, he was a fellow at the Weather Head Center at Harvard where his research included working on military implications of cyber capabilities. So I'm sure you can appreciate why we're excited for this panel looking at these issues of the intersection of law and technology from very different perspectives but yet very complementary perspectives. So I'd like to start with a broad question to the three of you and we'll go into questions for each of you. But we talk a lot about how technology and innovation often outpaces security. The question that I will ask each of you from your varying perspectives is how do we ensure there is a legal framework in place to support innovation and technology development while not inhibiting them but also promotes that innovation in a way that is structured effectively for the challenges that we're facing in this space of cybersecurity. Paul, I'll start with you. I don't know if you want me to start with Harvey. No. Start with Harvey. I'll start with Harvey. Great. Let me thank New America and you know what's a bit for it because these guys have been extraordinary in doing this. Full disclosure I was a former fellow at the New America and my remarks here today are in my private capacity and related to my cyber experience. I'm not speaking for an entity that I'm associated with in the government so I always need to caveat. So that question of legal framework is first, if you think of the opening remarks the opening remarks is an argument for the use of prosecutions to help generate norms. And that is not historically how we've often thought about norm creation but it is a very powerful form of norm creation. And as David pointed out, it was an unprecedented indictment. It helped change the nature of the game. In 2011, the National Counterintelligence Executive had released the Economic Espionage Report. It was the first time that a government report in an open source named Russia, China and Iran as advanced persistent threats focusing on the issue of the stealing of intellectual property. The government put that forward in 2011 but the question is what happened next is a legal issue. So what happened was the indictment which focused a great deal of attention. The other way we normally create legal frameworks is legislation. The legislation issue has as you know, who people who follow this, I've been involved in this since the mid 90s. We've had now a series of national acts to try to do information sharing, SIPA. But we have not really been able to penetrate the public private partnership to create a legal framework that everyone feels comfortable. And one reason is in the event in which you share information with the government as a legal matter, you may be exposing yourself to lawsuits and liability. This becomes a huge problem for information sharing for a corporate America. The other area where you usually generate this legal framework norm creation is at the United Nations. So as those of you who follow this, the United Nations Group of Government experts, the UNGGE for those have generated a series of reports but they've become stalled. And the reports have become stalled. There's been a general agreement and consensus for instance that you should not attack a cert when there's an event, sort of the emerging response teams. But the core issue which is why this has been a problem for the legal framework touches on what David mentioned which is the concept of sovereignty. And the sovereign question as to what the area of responsibility is and how you share information as a legal matter has proven to be extremely difficult given different approaches that the Russians have, we have and that the Chinese have. So, and I'll end with that legal framework as you know Supreme Court just granted Sir Sherry yesterday on the Microsoft case and the Microsoft case concerned Microsoft's position was that because the data was being stored in Ireland, they did not have an ability to access that data through what they believe is the current regime of the what we call the storage act. And that even though there was a fourth amendment that there's extra tutorial extension of American courts and power. That basic issue as to whether or not a warrant can be served on an American company with jurisdiction here to gather data and information stored outside of the United States. There still is not law on what the appropriate responsibilities are. That's what makes the space so extraordinary because of this level of the unsettledness of a whole variety of issues that we had settled domestically and are now litigating as to what effect will be internationally. And I'll end with that. Thank you. So I would build on what Harvey's just said and say that the simple answer to your question is to give more money to think tanks like this and universities like Pittsburgh to get on top of understanding what that framework looks like. Because it's not that we have an absence of a framework. In actual fact, there's a pretty good argument to say that that argument is that framework is being constructed by courts and by lawyers in a rather ad hoc way, largely because in this world which is increasingly dependent on information systems, the courts have to make decisions and those decisions have real world impacts whereas the sort of committees and institutions that have been established for internet governance and other things have yet to really establish that the framework's in a sort of overarching way. And as a result, we're sort of seeing things like the Microsoft case or indictments of the PLA-5 kernels set the framework within which we're working. The challenge is that the ad hoc nature of that often takes us in directions that don't necessarily take us to where we want to get to. So one of the challenges of cyber sovereignty that Harvey was talking about is it plays into some of the ways of thinking about the internet that favor authoritarian regimes and people who want control and potentially even cycle innovation. But even without that, failure to sort of get on top of the issue encourages governments to do things that may not be consistent with each other. So even if you have the best possible motives, you often get the law bumping into each other. So you have in California, the then, Senator General Kamala Harris saying that if companies don't adopt the 20 critical controls that were created by the NSA and are now run by the Center for Internet Security, then the state will take action against those companies for being negligent. Same time, in New York, you have the Department of Financial Services setting regulation for that sector. And so as a result, you have a sort of patchwork that's creating itself in real time without necessarily, wrong to think you could ever have a controlling brain, but without necessarily a good sense of whether all of the individual decisions take us to the world that we want to get to. So the question, as I understand it, is the world changes quickly, behaviors change quickly, technologies change quickly, laws change slowly, and there is bound to be mismatch there. So the question is, how do you put in place some sort of framework for making sure that laws approximately are appropriate for the technological and behavioral landscape that we find ourselves in, which is changing minute by minute. I have to say, I don't know. I mean, I'm an AI guy. I don't know much about law and I don't know much about cyber and so I shouldn't really be here. But from an AI standpoint, the whole question of how you design a law or a policy is a fascinating design problem. So think about what it's gotta do. I mean, it has to maximize the utilities of both the individuals acting locally and society as a whole. That's the first requirement of any policy. And so viewing this as a design problem or as an optimization problem, you have at least two objective functions. You have the objective function of the individual and the objective function of society. And that's just the beginning, the beginning of a tricky design problem. What bothers me a lot is, and again, I'm naive about this, but what bothers me a lot is the strong impression that the consequences of new policies are not analyzed in any rigorous way. They're not analyzed in any sort of model-based way. People sort of go with their gut or they react to events. They don't build models and try and figure out what will be the consequences of a new policy. To me, this is completely nuts. I mean, we don't design computer systems this way. We don't design anything this way. So I think that one answer to the question of how do you make sure the law is more or less in sync with society is you get a better law design process that is model-based so that you can analyze unanticipated consequences. I have a little example. How many of you think the self-driving cars will reduce the number of accidents on the street? Yeah, that's what I thought. Now, how many of you think that dividing the driving population into two parts, 20% which slavishly obeys all traffic laws and 80% that doesn't, will increase or decrease the number of accidents? How many of you think that will decrease the number of accidents? If you put 20% of the cars on the streets slavishly obeying the laws, you're gonna frustrate the other 80% of the people. These cars will drive at exactly the speed limit. They will slow down when it's a yellow light. They won't cross when there's somebody in the sidewalk. They won't go into a corner when there's someone in the sidewalk. And you're behind that car on your way to work. Why do I bring it up? Because nobody's thinking through the unanticipated consequences of this intersection of law and technology that's simply going with their gut. And so I guess my answer is that in order for law to keep up with society, you need better models and you need to analyze the hell out of these decisions rather than going with your gut. Harvey, as a follow on to some of what you discussed, so you talk about in listening to David's remarks, the use of prosecution to help generate norms, creating legal frameworks through legislation. If we follow on with something that Paul said regarding designing a process that is model-based for law, how do you see that working in the international context? So the challenge, as you rightly pointed out, is that we have a lot of global issues that don't have boundaries, but yet we have sovereignty and nation states that are operating in their own realm of law. And I was recently overseas and one of the issues that I heard a lot was, well, you can all talk about this from a United States perspective, but all the information we want is coming from US-based companies. We want the Google information. We want the Facebook information for our investigations. So how would you look at what a design process is that's model-based globally for addressing some of these issues around law and technology? Great, we're gonna thank Paul because we call this, why this institute is so important, the Geek-Wank Bridge. And this is a geek, the Geek-Wank Bridge. You represent the geek tribe. Oh, I am? Yes, absolutely. And I will demonstrate this by asking how many people in the audience can write an algorithm? Raise your hand. These are geeks, right? The rest are WANPs, which is the policy people. So we do have a legal operating system for how we do this and I carry it with me usually. And it's called the Constitution, which is really not an, it's an AI document in order to create what I would say total frustration for the policy process. The founding fathers clearly wanted to create an extraordinarily complicated issue that would not maximize utility of efficiency, but would maximize the concept of power and democracy. And why the AI world has an interesting issue when brought the way you frame the issue is because you never mentioned the term liability. And liability is what the economic school of the law tried to figure out, which there is an area of the law, which is law and economics, that tries to do exactly how you frame the issue to maximize utility on principles. But the problem is when you introduce fairness as a concept, it becomes a very mushy way of understanding how to go forward. So in your question, and let me, my heart goes out to you, because I'm a former dean. So being a dean is like having the ability to say kick me as I walk around the back of it. How long have you been doing this 10 weeks? Give me a break. So my admiration goes round to you. So your question is what we've decided is one way to think through how to move the levers on the international level is in the international criminal court, we have something when America goes forward, we have when we have troops on foreign territories, we have status of force of unit. So before we can actually deploy military troops on a foreign entity, we usually negotiate with that entity what the rules of the game are gonna be. These are the issues arise. If one of our troops involved in a criminal matter, how will we prosecute that individual? Will the host state have it? Will it be brought back to the United States for our activity? And these are referred to as Article 98 agreements. So one thought about it is when we usually have a breakdown on the international system about what makes sense for us to go forward as general norms, we usually then resort to bilateral agreements. So one way to think about it is David mentioned the five eyes which are the common law countries which are our allies where our friend at one point used to be a citizen of and I think now is becoming a dual citizen and that there are like-minded states and the argument would be that we would enter into bilateral agreements with Canada or England to create what would be an appropriate way for we to respond if we believe our networks are being penetrated. So David alluded to a big area of the law is hack back. How would you either allow the government or the private sector to protect its network and move forward? What is the norm there? So the theory would be we would create a series of bilateral agreements with like-minded countries about what the left and right margin should be for our ability to respond. And what David mentioned, we believe that I guess the argument would be one of the issues that brought Iran to the table to negotiate on the Iranian deal was our economic sanctions. So if like-minded countries start to use similar sanctions and are penalizing what we think are outliers and at that point what makes us very powerful is a reserve power of the United States is that we are still the international reserve currency in the international economic market. The dollar is still the currency that is used. That is given treasury through its power extraordinary leverage. So using those tools I would say is how we think we may move forward in the short term and then see whether or not we can go forward. And I'll reserve comments for why there's an interesting issue brewing about sovereignty and fake news and how that news goes forward because I would resist one proposition of your question. I do not see Google and Microsoft as American companies. They are world companies. They see themselves as world companies operating in multiple jurisdictions. So it's an interesting question though because of the data of companies that are US based is contained by US based companies. The issue then comes up to whom does that data belong and what the international countries are saying that data then belongs to the United States and we actually have to create agreements in order to extract that data. I absolutely agree that those companies are global companies but I guess my follow on question to you would then be what are the sovereignty rules around where the companies are based? This is Paul's issue that we raised. What's the legal framework? So David's a former prosecutor. We believe if an American company is here so we have jurisdiction over it and that company has information that is spread out over the world. We believe historically under the law when we tag that American company for its information for the lawsuit it's under a duty and obligation to go out into all of its entities to get the information to be brought back for the lawsuit. What if the lawsuit is occurring in another country? Well we'd have a similar issue is do we have jurisdiction over that particular suit but similarly if let's say England was going after an entity that was an English company that also had subs in the United States we would believe the court order historically would say that you'd have to tap all of your issues all of your entities to respond to that appropriate summons or the appropriate request through the court. That's where the sovereignty issue comes in. If a sovereign says because we have this jurisdiction we are not gonna respond, that's his MLAT problem. That's his expedition problem and that's where you're seeing the tension of what we think is appropriate legal processes to get the information and a sovereign saying no we're not gonna, that is a breakdown of what we understand of comedy of the respect of countries with each other to negotiate on privilege. Well we do have very powerful comedy is usually on child pornography. There's almost no state that tries to defend child pornographers. So we've been able to build very good exchange in the law enforcement trying to knock down those links. When you move into cyber and different elements of state interest you start meeting resistance based on a sovereignty issue that you don't see in other types of criminal activity. And so that is certainly an emerging trend with law and technology. And so Ian I would go to you as what do you see as the other emerging trends on this intersection between law and technology that are being crafted in this era of cybersecurity and particularly in a global context? So I'll offer two and this is clearly not a comprehensive answer but two things which I think are worth considering that build off the back of what Harvey just said. Firstly that and they're based around the sort of idea that what we're seeing is a sort of cyber physical convergence. So previously we were able to think of the internet as sort of something that's separate from our physical lives. Increasingly our daily lives have information systems sort of embedded within them both literally and figuratively. And as a result of that the sort of the highest level we're no longer really thinking about internet governance. We're thinking about global governance. So the way that the question is shifted away from what entities should govern the internet and towards how should states who are literally built off the back of the internet engage with one another where the internet sort of informs those relationships. And I think increasingly what they were saying is that those countries are realizing that this internet thing is too important to not be taken seriously. And that's driving us towards sort of a very sovereign approach which fits quite closely with the way that the law works but doesn't always align with the way that we've thought about how the internet works. And then sort of in a different way of looking at that we're also another trend that we're seeing is this clash of security cultures. This comes again back to the point that Harvey was making about liability. Like previously software industry has been able to succeed off the back of a philosophy of permissionless innovation. You ship early, you ship often, you fix your bugs when you have to and over time you start to develop the best products and the ones that have got to the most people develop network effects and you have a tech industry that's dominating the world. But if those technologies find their way into physical products like cars and things that have a very different security culture that's based on a precautionary principle where you wouldn't drive the car off a lot if it didn't have tires that worked and brakes that worked you have this clash and I think because now we these information systems are so embedded in our lives though that clash is going to be mediated in the courts ultimately and we'll get a sense of how important as a society we think innovation is versus protecting people from themselves. Thank you. So the question is as I understand it has to do with the future of law in an increasingly globalized environment, is that right? Yes. So thank you for the lesson on the constitution but I thought we were discussing laws. The constitution is a set of design principles that tell us we prefer laws that look like this to laws that look like that. We prefer encouraging this kind of behavior that kind of behavior it's design principles it's not the individual laws, the bill of law, bill of rights is powerful but what is the laboratory in which the natural experiment with laws is played out? The laboratory is municipalities and counties and states and countries and only by having all of those natural experiments going simultaneously do we have any hope of coming up with a well-designed set of laws? Remember, we're not designing laws the way a geek would design laws we're designing laws by throwing them out there and seeing what will happen and to get them right you have to throw them out there a lot, right? So I don't think we should close the lab at this point. I think it's too early to close the lab. I don't think we should be marching too quickly to one set of globalized laws even if companies are globalized and finances are globalized and information is globalized. I don't think we should be moving so fast toward globalizing laws because that's essentially closing the lab. Now the problem of course is that if you have different laws in different places then all of those companies and finances and one thing another will figure out how to exploit the differences but that too is a natural experiment. That too provides information and you might not like the consequences in the short term but it provides information. So don't close the lab so soon. Eventually you wanna go to global standards because we live in a global world but I don't think we have enough information to do that right now and there are a bunch of places where science and other things that we care about like policy have been brought to a crashing halt because the rules preclude it and the rules were poorly designed and prematurely designed. I can tell you that there's work that I wish I could have done as a program manager at DARPA that I simply couldn't do. I wasn't allowed to do it because there are rules in place and those rules were put in place prematurely or for bad reasons. Don't close the lab. Deal with the fact that people are gonna be exploiting the differences in laws and learn from that. I would say that. I totally agree with you. So what I love about it is that you've taken what we call the Brandeis position. So Justice Brandeis thought was very... He was a good guy. Yes, he was a good justice. That the argument would be is that there's federal law which is the national law but we have state laws and state legislatures and those 50 states are the experiments, the labs. And when they come up with issues that finally work that's when we should bring up the federal. So it's exactly that position. But the question which I think I don't wanna become moderate but is that from Europe the issue that we have when you said the trends. So one huge trend for the law is what your area is is artificial intelligence. That is generating a huge set of potential legal frameworks. Not just the cars but what also Ian raised which is that we have the issue of encryption. And encryption is also posing an incredible problem for us all states about how to go forward. And then we have quantum computing which also is generating a way. Especially about encryption. Especially about encryption. And then we have the issue of how we understand the role of surveillance and the technology involved in surveillance and what that allows the state to be able to put together when it puts its cases are issues that we're all starting to have to confront with. And the technology is generating it but it's generating it within our understanding of what we have to do for privacy and protection. So I wanna go back to the privacy and protection a moment because I think the encryption and looking at states and how we start smaller if we go global but Paul you talk about a systems approach and a polymathic approach and you examine what disciplines have in common. So if you're looking at this and in light of what we've just talked about what do science and technology and law have in common and how can these commonalities be used to model and manage this increasingly stressed system? And what we've done I think even in this discussion is identify how we're each defining stressors. How would you look at that and what would be your thought in moving forward? The principle of polymathy that okay so for a long time I thought the polymaths were people who had just mastered French and then music and then history and law and they just knew a whole bunch of different disciplines and it took me pretty much my whole career to understand that the reason that they were able to master those multiple disciplines is they'd figured out what those disciplines have in common. And so we should be teaching what disciplines have in common and if you think about it that's not how university works. Disciplines teach every subject as if they're the only subject and God forbid you should see any commonality with anything else. Oh, we're not like that. That's how we teach. So we at Pittsburgh wanna teach a curriculum that is polymathic in the sense of stressing what disciplines have in common. So now back to law and technology. The language that I use is probably has different nouns and verbs than the language you use but I think the concepts probably aren't very different. Most of us understand that behavior is composed of actions and that actions have payoffs. And some actions, well like almost all actions are legal and almost all actions have payoffs that are determined by markets and things like that. Some actions are prescribed and those payoffs are established by law although even there are differences in the payoffs associated with actions. And a person, irrational, irrational doesn't matter but a person acts and in consequence of those actions receives some sort of payoffs. A rational person would act in such a ways to try and maximize those payoffs and that would include occasionally breaking the law. Especially when you understand that actions and their payoffs are stochastic. It's not guaranteed that you'll be caught and it's not guaranteed that you'll be sentenced and it's not guaranteed that the sentence will be a bad one. Use a stochastic to some extent. And so a rational person would set this up as a sort of a fairly standard sequential decision problem. I can now write down the formula for the sequential decision problem and I can describe the algorithm for solving it in an optimal way. And I could even say that's what people are doing on a daily basis if I was talking to my friends in psychology or behavioral economics. And I'm describing stuff to you that everybody should know. I'm describing technological concepts and mathematical concepts that I think are a pretty good model of how people behave. And if you master those concepts then presumably you would be able to approach law from that grounding and also approach self-driving cars from that grounding. I mean, there's just a set of concepts and I've just described eight of them or something which everybody should know because they undergird so many things. Roughly speaking, maximize your utility by optimizing a sequential decision process. If you don't know what that means it's because you haven't learned the technical language, the underlying concepts are absolutely obvious to anybody and they underlie both law and the design of virtually any object. We should be teaching people to think this way. Don't get hung up on the terminology. Think about the underlying concepts that unite fields. So in light of that, and before we get into questions I wanna ask one final question to the three of you and I'll start with you Ian. So in looking at data not being in one place, and looking at who has jurisdiction over that data and what Harvey just started to talk about with encryption and privacy. And we discussed, is there, and you talked about the laboratory of smaller entities. When there was a law or a piece of legislation that was just brought up to the Hill recently on IOT standards for driverless cars. There was a lot of kickback in generally speaking because there was a concern that well if you start with driverless cars, if you just do driverless cars we're completely forgetting that we actually need to be looking at this head-on around all IOT devices. We can't just start with a segment. But to the point being that we have to take this off in bite-sized pieces and we have to prioritize. So if we look at states and you talked about the California data breach discussion, New York sector on financials. You saw the states, the attorneys general pushback on Equifax when they said, you've waived your right for a lawsuit. Are states the right laboratories for doing this? And should we be looking at states to start piecing together the key priorities? And then to Harvey's point, do we elevate that to a federal level? Or because data doesn't belong in any one place and we're still figuring out who has that jurisdiction, do we need to be thinking? I think to my first question to Harvey of a more global approach, is there a hybrid here? But we clearly have these multifaceted issues and depending upon physically where you're standing as well as theoretically where you're standing, you're gonna look at this very different. So how do we start, you know, fighting off these pieces? And I think I was just, it was notable when you saw the pushback on the driverless car because for many people, it's like, wait, we're actually doing something. But there was the concern that is this gonna be enough and are we gonna get too stagnant or impeded by just focusing on one rather than the whole work? So I know you do a lot of work with the state so I first start with you on that. As a non-lawyer and a relatively new American, I'm probably the least qualified person in this room to answer this question. But I think what is certainly true from looking at the way the cybersecurity legislation and regulations are developing or being developed to the state level is three things. One, there is real opportunity to innovate at that level and certainly there is some opportunity to do things that can't be done in Washington, partly as a result of the fact that the political process is particularly gummed up. But second, not every piece of state legislation is very good to say the least. And that's partly a result of the fact that they simply don't have the results or the resources to understand the full consequences and sort of complex interactions that exist at Paul's point. And thirdly, that there are opportunities to do things at the state level that don't necessarily involve legislation. And this is where the sort of voluntary framework like this framework came in. And so one of the sort of challenges going forward, I think is, and this is not a new insight, but is understanding where the opportunities do sit at the state level and where they best don't. And I think it is fair to say that not enough attention has yet been applied to where the right, at what point, which government should be taking forward particular pieces of legislation, where legislation is required, where it's best left to voluntary standards. So I would sort of respond the following way. Why the space is hard is because you have four areas of vulnerability. And the first area of vulnerability is the software. So software is written that has zero days in it or have a whole range of problems that we have created a market that allows us to go onto the marketplace which is a major vulnerability for people who have that ability to understand how to exploit it. The second vulnerability is the hardware industry. So we have a huge supply chain problem, which is when we produce all the things you're holding in your hands and are looking at right now and are communicating with, you have no idea what else is in that particular product that allows entities to have vulnerabilities in it anywhere from ransomware to monitoring your activity. The third problem and vulnerability is what we affectionately refer to as carbon units. You know them as people. And people inside are threat problems. You can have the best software and the best hardware. But if you have an individual who's willing to place something on the network to allow it to become vulnerable, you're vulnerable. And then the fourth vulnerability is, we all ride on internet service providers. So the ISPs are extraordinarily powerful and if you notice there's a great deal trying to focus on the ISPs in addition to the social media as how we can get an understanding of how we can then participate in creating more of a secure environment. States just don't have that power. The problem set of those vulnerabilities require a whole different array of ability to sit down to improve each one of the areas. It's why we always say offense beats defense at this point. Because we do not have strong enough defensive mechanisms that build in the certainty. Again, way beyond, a nuanced promise that will not respond to a hammer, which is what legislation always is, versus a rifle shot, which is what a law case is or a legal case until you get a network of liability and trust in exchange. So I think one has to think much more about where you can have leverage and the other aspect is I'll end with, when you think of the private sector, we actually don't have a private sector per se. We have a set of regulated markets that have different types of sectors. So the banking sector has its own regulatory market. The industry that deals with energy, the FERC, the Federal Energy Response Commission, they have their own regulation. So using those regulatory entities to try to raise the bar for what they're able to do or not do with software and hardware, what rules we put in for the carbon units, which is why we say privacy is ending. Because of every one of you when you click onto your networks, I bet it's true in the American Foundation, you say yes to that first screen. And that first screen says that the network has the ability to monitor exactly what you're doing and doing in order to find out whether there's a problem with the way you, as a user, that user agreement. So it's such a multi-layer issue that we have to have. I think there are other ways of getting leverage, which is including the market. So tax codes, insurance, cyber insurance, that's a way of getting this on scale to resolve the problem. The legislature is only a small part of understanding market. And with the following, we've created two markets that are very efficient at this point. One market is the entire cyber and it related what you do for Microsoft, what you do for Google, for social media, incredibly effective market. That's incredibly vulnerable. And now we've created another multi-billion market, which is the cybersecurity market. And the cybersecurity market then tries to fix the first market. So both markets are multi-billion dollar markets, but they are not creating what you would say utility and efficiency in the long run. Maybe they will, but currently those two markets are creating us and having us vulnerable. And it's unclear what legislation can do in relation to those two incredible market systems. So the only thing I would add before hearing from Paul is it's interesting, we've talked a lot in this context around industry, but when you look at elections and you look at that information from a state-based level, I think we would, what we're hearing is that while DHS may be saying states you should come to us, DHS really doesn't have any authority over how states run their elections. And so when we are now in an era where protection of critical information is becoming more important than protection of critical infrastructure, how do we look at that in a global context, but then how do we look at in a national context and what is the role of states in that capacity? So Paul, I would just go to you for final comments before we open up to questions. I was most interested in Ian's comment that there are some states that have very poor laws. And the reason I was interested in that is this is another one of these ideas that everybody should be familiar with. If you look at any machine learning algorithm and you look at its performance over time, you see a very rapid rise in performance and then it starts to flatten out. This is true of all machine learning algorithms and there's some sort of fairly standard statistical reasons why that would be. The most important of which is that failures, as you're learning, failures become less and less frequent. So if you learn from failures, it's natural that the opportunities to learn would start to, they would become less frequent. So I was actually fascinated that there are still some states that are doing things really badly because those are learning opportunities. And if we don't have enough learning opportunities, we don't learn. So yeah, I'm all for keeping the lab open and providing as many learning opportunities as possible and thank goodness, some people are doing things badly. I think that the trick is fine, and I certainly wouldn't like to point it to any particular states, but I think part of the challenge here is having the institutions and the people who see it as their role to learn those lessons and to do that monitoring. There's a reason why states do some things badly because it's an incentive system for a variety of interests that are benefiting from it. And that's always the issue. If I'm more of an optimist and I would say the reason for it has more to do with resources and how things are organized and where the capabilities lie, I think, in those issues. None of you is saying that these are not learning opportunities. It's always a learning opportunity. And with that, I think we'll bring up another learning opportunity and have Dave Hickton come and join the panel and we'll open it up to Q&A from the audience. So we'll start with questions, yes. So much, my name is Ashwita. I'm currently an intern at the Information Technology and Innovation Foundation. So my question to you is when it comes to the trade-off between privacy and security policies and permissionless innovation, as you input it, is something that each government probably considers and makes on its own. However, in the current situation, when it comes to jurisdictional issues in data, jurisdictional issues over IoT devices or the internet itself, do you think that it would be incredibly easy for global companies to default to the most restrictive, such as, let's take the case of the GDPR, maybe the Northern European companies or Northern European countries are in a good place technologically to impose those. But maybe in countries like India, we need to prioritize permissionless innovation more. Do you think that this would make it easier for companies to default to the most restrictive policies and how do we fix that? The answer is yes. And the example would be is that when you're advising a large corporation and you either go to the top or the bottom of the barrel because you wanna be able to have uniformity. So usually the entity that is the most, let's say reporting, what's required to be reported. What constitutes an incident for reporting? There's a whole range of what that would be. So the lawyers fight about is an incident that requires reporting if it goes outside the company that someone else have to know a third party. So whatever that rule is, you have those rules and then you will advise to set those rules because it's more efficient. But to your question, one of the more fascinating issues is who owns your data? So part of the European approach is that maybe you should own your data. When you sign off on most of all your applications, you are usually, how many of you actually read the agreements that you're clicking? Only one person, but I know him and he's that plant in the audience. So most people do not, including our panelists don't read them. And usually you're basically telling them they can use your data for a variety of purposes. But if it required that they would have to pay you a certain amount in order for you to give them that data, because you own it, they don't own it, that will change the dynamic dramatically because whether one likes it or not, it's all about the data and how you get control of that data, how you rack and stack that data, how you exploit that data, how you use AI on that data, how you protect the data. So at this point, individuals don't own their own data. If Europe is moving to that, if Europe moves that, that will be a game changer for how what the rules are, for what is allowable and will require you specifically to say yes or no to that. That will be quite fascinating. Hi, my name is Ji Ping-Wong. I do the data analysis, big data. I'm surprised. I mean, nobody talked about identity set today. She only mentioned about the equal facts. And oh, no, equal facts, 145.5 million record of us. And this is not the first company impact. We will not be lost. So I especially agree with you. Law is behind the technology. And I think this problem is social effect over us. And right now, what Congress does? Just call the hearings. Does it work? Call hearings for equal facts? Call it before target home depot? So wait, wait, wait, wait, wait. That approach does not work. Another company will be hacked. More consumers' information will get lost. We need to think about differently, how to prevent it from happening again. How do we think about it? Think about why those bad guys steal our data. How do they work? In this way, we can come up with a very simple solution. Just think about this way. If bad guys steal my data, they cannot use it. Which is useless? Will they steal again? To steal yours? They already know they cannot use mine, but they will not steal his. So that's the solution to totally solve this identity set problem. This is not just a reaction. We need to push federal government to set some rules. Actually, I already have a solution. Very simple, cost consumers nothing. Zero dollars. If you think about it, one hundred forty five point five million. Each one of us spent ten dollars. It is more than one billion. Plus, the business government agencies every year waste more than billion dollars. Probably, I don't know, multiple billion dollars. So it's an interesting technology and innovation question, right? In a law question around identity management, which I think Paul is interesting for you, and I'd love to hear from Dave around the law behind that. Could you make it a question just because we're running out of time and there are a few other people who had questions? Well, this is not a question. It's a statement. What I think is, of us, especially the bar association of American Army, the lawyers should work together to push federal government to set up some rules so that our PIIIs will be used this to the set so that they don't have a motive to steal again. Thank you. Thank you very much. Paul and Dave, do you want to talk about just the issue around the religious actions? Well, sure. I think you can talk about this on simple terms like this. It took us a long time to figure out that we could put exploding blue paint on dollar bills that were stolen in bank robberies. And we probably could figure some way out if we can come up with dual-factor authentication of misuse of your material will have some similar consequence. If you want to look at it more complicated in terms of what the nation-state norms are today and what they might evolve to, Harvey talked about sanctions. If someone uses data that is taken, particularly if it's a state-sponsored attack, then there will be serious sanctions. But it's really a question when we evolve to these laws about what trade-offs we're willing to make. I'm told all the time that we could reduce the cyber-hacking threat by increased cyber-hygiene and through methods that would largely not be too complicated. I'll mention the prime three, dual-factor authentication, segregation of data, and requiring members of a workforce to make a phone call when they receive an attachment as a protocol. And the question for all of you, particularly those of you in private industry, is are you willing to fire workers who don't follow those three guidelines? And because you have to accept whether we call it legally owning our data or the data is in the commons, we all are losing billions of dollars every time there's one of these hacks. Do we have the will to impose these sanctions on ourselves by cleaning up cyber-hygiene? And if we have that will, then what is the next step we're going to take for the next wave of a threat? Could I amplify on that a bit? I think we've been focusing on the downside of privacy. We've been focusing on the threats to privacy, the consequences of getting hacked, but there's, I think Dave's point about looking at the risks and making a judgment about what a risk worth taking. I think of that point extends to the idea of data as a public good. So we've been talking about data as my data, I suffer, I'm hacked. This is bad for me. But let me draw an analogy with vaccinations and vaccines. You vaccinate your kids, and that's actually a bad thing for you to do from a personal standpoint, right? Those vaccines could kill your kids. They kill some kids. But you do it as a public good. You recognize that it's a public good. You don't have the same attitude towards data and your data. And yet your data is every bit as much a public good. And by making that data available, you expose yourself to risks. It's very important to keep Dave's point in mind there are risks associated with making data available, and they are risks to you. But there is a public good associated with doing so. I am alarmed that so much data is owned by private companies. It isn't a public good at the moment. It should be a public good. Imagine how medicine would benefit if you would make your medical records available. So thank you for the lesson in Justice Brandeis because my favorite program at DARPA was called Brandeis. And this is one of these places where law and technology really do come into contact. The Brandeis program was one in which you would be able to say how you want your data to be used and it would be guaranteed to be so by a variety of algorithms that have to do with proving the security of the data. Interesting, right? Because laws will change if that technology in fact worked. Privacy laws would change and there would be public good consequences. So this risk balance, this risk balance associated with data can be changed by technology. That's I think one of the really interesting places where technology and law come into contact is in changing the risk balance. On the geek wonk gap, because I think I look back on the guys like Benjamin Franklin as being like a great geek wonk gap guy. Two comments real quick. One on Ian's comment on the cyber physical relationship here because I think this has not paid enough attention. I spent four decades in technology and one thing I found out is stuff that I was working on invented in 1970s is still out there and is still basically working the same way. And we're in this process now releasing a ton of stuff into the cyber physical space that has a life cycle associated with it. And it's not as easy as buying a cell phone every two years. Now it's going to be embedded into everything. And I think one thing we need to get out there, particularly as we kind of do this regression analysis of how we change our laws is how we take into account this whole life cycle aspect of this because I think it's not a problem that I'm seeing being addressed at all. Everybody assumes that we'll be able to turn the switch and everything is compliant and it's not the way it's going to be. So I think somehow we need to get that into the conversation and quite frankly the only way I see it happening is getting more technologists out there to do that regression analysis along with the policy people. So my question is, where does this start? Does it start at the university? Does it start in the existing kind of public space? How do we get a handle around this because I don't see that conversation happening yet. Something to that if I may. We in New America, as I said at the beginning, have sort of working in partnership with Florida International University in part and very pleased to work with University of Pittsburgh and other universities who are starting to think in a multi-disciplinary way around this. But the learning process that we have gone through is that it is not in any one place that you're going to be able to make this change. So there is undoubtedly a shift, I think, required in the way in which universities do their business because, as you say, these issues are by their nature multi-disciplinary but they also require an understanding of how things work in the real world as well as on the academic side. So more kind of apprenticeships, for example, will be really important there. But that obviously needs a pipeline so it takes you back into the sort of K-12 system which is tricky because these problems are recognized at the federal level particularly but education policy at that level is set well below the state level and so bridging that gap is going to be really important. And then there's the fact that even if you get everything right with the universities it's going to be a time lag before the people who make decisions are going to be in a position to make an impact and that implies a process of continual learning, adult education, whatever you call it, training on the job training for which the universities may well be the right place to do it but it starts to change their role and how they engage with their communities and I think Pitt and others who are beginning to think these through have a real opportunity to do education in a different way and cyber is actually a pretty good way in which to get to some of those issues. So when I was in high school our major was called Latin Science. The theory was you didn't separate science and the arts. So CP Snow wrote two cultures which is when that separation happened in the academic community so students started to get streamed out so if you were not, when I was in graduate school, you required to take statistics but it was considered a foreign language. You could either take French and German for your two languages or French and statistics and it was a demonstration of how the community clearly was not understanding what would be considered and required so I'm very excited about your program but I thought when we were involved in that one of the ways you'd see a lost student would also have to understand as you're going up to your system that if they're going to go in this space they should take a computer science course. They should start taking these cross educational for this space. On your question, the DOD has this problem. We have multiple problems with huge legacy issues. We call that the supply chain problem and that supply chain problem is how you get leverage. That leverages, we have something called the defense industrial base and there are core companies that are the top 10 or 20 10 in that base and then how they filter through through the subs. So we are trying to figure out how to get leverage on that problem because the legacy problem would be is dramatic because if it turns out that we have systems that when you ask it to do X it does Y this will be a huge problem and you're evolving the reality or it doesn't do anything. So there's a real focus on what you put your finger on about how to go forward to create new systems and figure out what our legacy systems to separate what the vulnerabilities would be in that space. Well, we in academia and the think tanks are working on this and we're not engaging in naval gazing, we're doing serious research that this is not a new problem. If you go to pop culture war games with Matthew Broderick was out in 1983 the matrix was 1999, some of you in the room probably weren't born yet. If you go to policy Leon Panetta and Bob Mueller in their former jobs said we may need a cyber Pearl Harbor or cyber 9-11 before we get serious about this. They said that a decade ago and we've had multiple cyber Pearl Harvors the most recent one being Equifax. And so I just think all of us who are interested in this topic have to up our game and up the speed of our delivery. I'm very excited with Paul Cohn's arrival at Pitt because we are working, we resist the term cyber security and we resist the term Big Data in terms of the commons on one side which is how we can leverage the opportunities of the internet. And we're talking about the law and policy gap on the other side and how we can make sure that the laws accelerate to catch up with this emerging technology. But we basically put ourselves on a communication system that 25 years ago only 3% of the public used and now virtually everybody uses and we're building it as we fly the plane and we need to really get serious about it right now because just based on my experience in my former job the threats that we face could completely take us down. Israel's created a fascinating model but it also requires military and public service for all their citizens but they basically by high school narrow through games who their top cyber talent is and they take those kids and they ship them into their cyber brigades to be educated for 2 years and get the latest skills and then those kids then go on and become the basis of the Israeli entire IT community. They punch way beyond their weight given their size of country. The government has used a couple of 100 million dollars and that's blown up that particular sector to billions and billions of dollars. We have not had a similar national strategy to figure out how to exploit the people coming out of Pitt or Miami to be evolved in a governmental entity. The government cannot compete with Google and with Microsoft for a competitive advantage. It's been a huge problem that we need something of that nature to orchestrate and organize that ability. Let me add to what Harvick said. The current statistics are that in 2018 we need approximately 1.4 million new cyber professionals in the public and private sector. For all of the young people in the room this is a great opportunity but we have about a third of that in the pipeline and that gap between what we need and what we have is growing. So I'm affiliated with the Air Force Association and we did a cyber camp in Pittsburgh with 211 high school in July. And there's a cyber patriot program that's part of this organization where 4,400 schools are running cyber patriot competitions and we aim to get all 45,000 high schools in the country in this competition where the students get scholarships. But we are well beyond what we need for our national defense and what we need in the private sector. I appreciate everyone's patience where you've run over. I just want to take this opportunity very much to thank the panelists Ian Harvey, Paul and Dave Hickton and thanks to all of you for your participation.