 Good morning everybody. Welcome to those who were able to brave the rain. I know that we'll be expecting more, but we want to get underway. My name is Nancy Lindborg. I'm the president of the United States Institute of Peace, and I'm just delighted to welcome everybody here this morning. For those who are watching online, you can follow us on social media with at USIP. And for everyone, especially since we have innovation and technology in the program, you can tweet using hashtag rule of law tech. And I encourage everyone to check out the new USIP podcast at usip.org backslash podcast. That will include today's event, as well as other programs that look at similar topics. For those of you who haven't joined us before, USIP was founded by Congress in 1984, dedicated to the proposition that peace is very possible. That it is essential for our national and international security, and that it is very possible. And part of what we do is work on the possible, on taking good ideas, research, innovation, and translate that into policy and into practice by working with partners around the world to find out how to make peace possible. And one of the things that I think all of us see over and over again is that in countries that are convulsed by conflict or stuck in those spin cycles of conflict, what they are all uniformly missing is a state society compact that assures the citizens that they can rely on security, that they have access to justice, that there is a rule of law that enables them to have confidence in their government. That's really the core. Every conflict country is different, but they almost all uniformly share that characteristic. And we therefore established what is called the IMPROL project more than a decade ago. And I hope many of you are either watching or with us here today. It's now an online global community made up of more than 3,000 rule of law practitioners from 100 and 20 different countries and 400 organizations. And this is an online community working together to understand how better to improve rule of law in conflict affected environments. So we continue to work on these issues as well through projects with peace tech labs and with our global campus online platform which will host a rule of law course later this year. So all of that is to say that we are delighted to be co-hosting this event today with Justrack, the justice sector training research and coordination program. And we're delighted to partner with Justrack. We share a similar dedication to promoting rule of law and to looking at how to reform rule of law in conflict affected countries around the world. We have adjoining missions and we are natural partners. This is the second year of the partnership. Last year we convened to look at some of the very tough issues that practitioners face in the field. How do you deal with the lack of political will? What are the incentives for rule of law reform and how do you deal with often very limited resources? So I'm happy to be reconvening today with our partners to look this time with the view to the future. Especially at a time where you have a lot of focus on the data breaches, on privacy concerns, we've just had the Facebook hearings. We don't want to lose sight of the fact that technology is also an important tool for good and how do we leverage it to accomplish the rule of law goals that we all share. All of these topics, innovations in law enforcement and how social media is changing the relationships between citizens and governments are essential for understanding how we can use technology to build legitimate, inclusive and accountable institutions that build justice and security. These are the heartbeat issues of our work. So before I introduce our co-host for today, I just want to give a quick shout out to Senator Lindsey Graham. Justrak is housed at his alma mater, University of South Carolina. He was instrumental in launching the rule of law collaborative that oversees this. And importantly, he's been a champion and a voice on the Hill for these kinds of issues. So a shout out to Senator Graham. And with that, I'm delighted to invite our co-host, Hamid Khan. Oh, you're going first. I'm so sorry. With that, I'm even, well, let me just also say I'm very happy to be working today with Hamid Khan, who is an alum of USIP and the director of Justrak. And himself has an illustrious legal and academic career. And so let me then pivot and welcome Harry Bader to the stage, who's the acting director of the Innovation Lab at USAID and doing excellent work there, moving technology and innovation into the development field and particularly looking at many of these issues. Harry, it's wonderful to have you here. Please welcome Harry to the podium. Well, first of all, I'd like to thank Nancy and Hamid for convening us today on this important topic. One of the things I'm going to be focusing on today is the role of USAID and working in the technology sector to help combat corruption. Corruption weakens institutions and undermines stability. It has a corrosive impact on the rule of law and erodes citizen confidence in the ability of their government to deliver essential services. The emerging digital economy gives us a new opportunity to use tools to thwart corruption. But as Nancy mentioned, technology can be a double-edged sword. But what we're working on at USAID is to build a global digital infrastructure that is affordable and accessible along with mobile services to provide unprecedented opportunities for more transparent and efficient systems that can build the role of civil society, that can foster democracy, capital economy, and the ability for individuals of freedom of expression. The four key things that USAID is working on within the Global Development Lab is on digital payments, distributed ledger technology, artificial intelligence, and digital identity systems. At USAID, we believe that supporting the development of an inclusive digital economy is essential with our partners, both in the field and other donors, for the progress of humanity to realize their aspirations around the globe. Digital financial services offer us the opportunity to build a transparent environment that can save millions of dollars in the process. Every year governments around the world pay out billions of dollars in cash. These payments include, among other things, government salaries and pensions. Payments in cash are incredibly difficult to trace. They're insecure. They're insufficient and are susceptible to leakage from corruption. When governments make the switch to digital payments, they realize huge efficiencies. In the Philippines, for example, USAID has helped two different provinces digitize their payrolls, reducing disbursement costs to the government itself. In the case of the Philippines, those who have adopted digital payments for their government workers have reduced administrative costs by 90%. In Afghanistan, when we helped digitize payments of police without any greater expenditure on the part of the government, the police saw their salaries miraculously increased by 40%. That is because we removed the vectors of corruption that were in the cash pipeline. So not only do you increase government efficiency, you also increase reliability and accountability. And since digital transactions are inherently more traceable than cash, there is also the ability to improve our processes for tracking illicit financial flows. This helps not only the United States government but like-minded partners to combat cartel activity, money laundering and terrorist financing. Distributed ledger technologies, as most people know as blockchain, is also helping, we believe. And we're working very closely at USAID within the lab to find applications of blockchain to international development. We believe that by increasing the number of individuals and organizations with access to these records, that these technologies can enable greater visibility and can therefore reveal attempts to delete or tamper with data. We believe that more audible records that may present themselves through this new technology is another opportunity to fight corruption. Likewise, artificial intelligence can help financial institutions more efficiently identify suspicious transactions and instances of money laundering. Again, artificial intelligence is one of the priorities of the U.S. Global Development Lab to see how can we use artificial intelligence in the development sector. But again, it's a double-edged sword because it's only as good as the people who designed them. And if you can use artificial intelligence to build an inclusive digital economy, you can also use artificial intelligence to build an exclusive or oppressive digital economy. Through our reg tech for regulators accelerator, USAID is supporting countries such as, again, the Philippines to develop prototype chatbots that can help in this process and thus improve accountability. And then there's digital ID systems. Again, something to be used for good or for bad. Comprehensive, well-regulated digital identity systems provide the opportunity for citizens to have access to formal services like business registration and land titling. These systems can help create a basis of trust and inclusion that strengthens democracy and free markets. In Liberia, USAID has supported the development of a biometric ID system to aid the Ministry of Education in its personnel management. The system exposed widespread corruption leading to the elimination of scores and scores of ghost workers. And so doing not only did it save the government Liberia money, but also broke the bonds of corrosive corruption in this case to help improve the ministry's responsiveness to the public. As we said, digital technology is revolutionizing development around the world. We don't have to look any further than here. For example, it was just mentioned the peace tech accelerators work to scale innovations focused on accountability, transparency and anti-corruption to fight conflict. And this has enormous potential, but the potential for good is equal to the potential for evil. And again, as we build a global digital economy, we have to talk about the enabling environment and how these things are applied. We want the digital global economy and infrastructure to build the capacity of civil society. We want it to expand the opportunity for economic capitalism. We want to foster democracy. We want to create a platform for truly free individual expression. But the digital future that I've just described is not preordained. There are other models that are being championed, championed by China, Russia and Iran. That is why the U.S. stands firm with its allies and like-minded partners to build a global digital infrastructure that sets us free. Free from corruption, free from want, free from ignorance, free from pestilence, and most importantly free from tyranny. So I welcome today's meetings and roundtables to discuss how can we use technology to expand the most important thing and that is the ability to free the individual from these vicissitudes of violence, conflict, corruption, poverty, and ignorance. Thank you very much. I look forward to hearing more about the work that's going on. Good morning. My name is Hamid Khan. I am the deputy director of the rule of law collaborative at the University of South Carolina. I am pleased and very heart-worn by the fact that everyone is here. This event is being live streamed as well and recorded for posterity, so please don't feel that the lack of presence in the seas necessarily negates the impact of this message. As part of our ongoing commitment with the U.S. Department of State in a cooperative agreement known as JustTrack, we have convened a group which we hope today will bring together both policymakers, scholars, tech individuals, and savvy individuals who care about the development issues across the world. The rule of law collaborative based at the University of South Carolina is collaborative in its nature that it brings forth both a variety of experts, academics, any university setting and stands unique among institutions across the United States. JustTrack seeks a similar goal, which is to bring about the inter-coordination of our federal government, our counterparts in academia, and various disparate actors to bring together both knowledge and understanding on the challenges and issues facing the justice sector community. In addition to the event today, the JustTrack network, which has already been established over the past four years, combines a variety of different trainings in a number of different fields and comparative legal systems as well as introductory understanding. We also provide resources for research on challenges and issues that occur across the board and then there are events like today, a symposium aimed at bringing about scholars, government, private and public actors on the issues that challenge us. And today, this marks the third in our innovation series. Innovation and rule of law and innovation in general have always been challenges that we face, but today it's also uniquely combined with the issues and challenges of technology. As has already been mentioned by both Nancy and Harry, this can be a double-edged sword, one in which we confront both the potential opportunity to reach out to more disparate populations in more far-flung areas of the world than ever before. And as today's event will be broken down, there are three primary pillars that we will focus on. The first being on law enforcement and the advantages and perhaps some of the pitfalls that occur within the law enforcement community and the use of technology. The second will look at engagement. Whether it's by through our phones, social media, or the like, we will bring together a host of actors to look at how social media and mobile apps have moved the level playing field to an even greater degree involving greater populations, greater interactivity, and their involvement. And then finally, the last panel today will focus in on e-government. This idea that government should be at the forefront and their engagement with populations has become a new and lasting challenge for both governments that seek to enhance their ability to reach their populations as well as those who are on the recipient end. And all the while, we are still confronted by the perennial challenges that confront rule of law, access to justice, fairness, equity, and the host of other resources that are related to the rule of law challenge. In addition, because of our gracious host at the U.S. Institute of Peace, and which I am proud to call myself an alum, we also have the opportunity to actually directly engage with some of the tech savvy individuals in a variety of different development spheres. And as you look towards your agenda, please take note of the fact that after this first panel discussion, we invite everyone that in addition to commiserating over coffee to engage with some of those who have graciously donated their time to talk about their work and to give examples of the application, including the work here hosted in coordination with the U.S. Institute of Peace, the Peace Tech Accelerator. In addition, and unusual to D.C., we're also offering lunch. And in that regard and in that spirit, we also want to offer the opportunity to hear from the remarks of a former U.S. ambassador who has been both wedded into the issues perennial to the challenge of law enforcement but is also cognizant of the challenges of technology. So today we will have a very full day. We will have multiple opportunities and we invite you to both participate in our panel discussions which will not only engage the various speakers who we invited from various agencies, public and private sector audiences, but for you, the audience to join in on the discussion, to ask the relevant questions that need to be asked and to push forward the substance of knowledge. Again, thank you to the U.S. Institute of Peace for serving as gracious hosts and we welcome you. Thank you. Good morning. Can everybody hear me okay? I'm Lindsay Freeman. I'm a lawyer and research fellow on the Human Rights and Technology Program at the Human Rights Center at UC Berkeley School of Law. I'm also a member of the Technology Advisory Board of the Office of the Prosecutor of the International Criminal Court where I used to work. In line with today's symposium theme on innovations and new technologies in rule of law programming, this panel is focused on enforcement. Innovations in law enforcement and in particular the obstacles and opportunities of new technologies in the fields of digital investigative analysis and forensic science. We have three very impressive experts here who will address the emerging approaches in law enforcement in forensic science and digital evidence. So we have first Joe Varani is a digital investigative analyst in the cyber crime lab of the Department of Justice. He provides digital forensic support to federal prosecutors on cases involving computer crimes, intellectual property crimes, and other crimes that involve digital evidence. Before joining the cyber crime lab, Joe worked in IT security at the DOJ. We also have Mark Grants who is assistant to the special agent in charge of the U.S. Secret Services Washington field office. He's been with the U.S. Secret Service for 18 years and is 24 years in law enforcement. He's overseeing high-level international investigations, managed intrusion detection and video surveillance systems at the White House and other protected facilities, and is presently the supervisor of the Washington Metro Electronic Crime Task Force. Finally we have Mark Mogul who serves as the deputy assistant director for special operations of the Department of Justice's international criminal investigative training assistance program. He's coordinated forensic development programs in Europe, Central Asia, Africa, and the Middle East which support investigations related to corruption, human rights abuses, cyber crimes, sexual assaults, trafficking, and other transnational crimes. Prior to working with the Department of Justice, he served for 14 years with the Pennsylvania State Bureau of Forensic Science. And so before we get started each one will present and then we'll have some questions mixing in some follow-ups that I have along with audience questions. I just want to highlight a few themes that arose when I spoke to each of the panelists beforehand. So one is the dramatic increase in the volume of digital information and evidence. There's been a proliferation of digital devices and they're collecting more information than ever about our daily activities. This volume of information obviously presents great opportunities for law enforcement and investigations, but it also has huge challenges of how you separate the signal from the noise and effectively get to the information you need for your cases. The second is the pace of change of new technologies changing. The rapid transformation of tools and for government in particular keeping up with every new tool that comes out and having the regular training so that they're trained on new technology and keeping pace. The third with the rise in concerns of privacy and security, new technologies have increasingly effective encryption technology. Encryption is now a default for many devices and a lot of new technologies make that digital material increasingly ephemeral. So there's concerns about how you identify what is important, how you collect it and preserve it. The fourth is working in cooperation between agencies, which is a challenge as well as cooperating internationally. The field of cyberspace and the international is a borderless by its nature and so there's challenges with working with international law enforcement in addition with private companies. Obviously the field of technology has raised many interesting issues out of the role private tech companies play, what their responsibility is and how they interact with law enforcement. And finally there's an increased expectation among judges, lawyers and juries of how you can scientifically prove crimes using digital evidence and explaining when it's more of an art versus a science and how conclusive you can be particularly with attribution, which is difficult. So with that I'll let the panelists speak and we're going to start with Mr. Varani. All right, so my name is Joe Varani. I work in a digital forensics lab in the Department of Justice. It is the cybercrime lab and the cybercrime lab is part of CSIP which is the computer crime and intellectual property section of the Department of Justice. And so CSIP is a group of about 30 or 40 attorneys and they have different focuses either on a computer crime team or an intellectual property team. They are focused on policy, some deal with litigation. They work on international coordination and training events. They are the contact people for the 24 by 7 network so if there is an emergency request that comes in from international request then they are the ones to respond to it. And then also as sort of a self-contained group in CSIP is the cybercrime lab. So the cybercrime lab is a group of non-attorneys that has a different focus than a lot of forensic labs out there in that we are dedicated to the needs of the prosecutor. And so normally when you think about an investigation and a trial case all the way from start to finish, you have a lot of work that goes into it up front from the investigating agency. They do a lot of research and investigation and they get their search warrant and they are able to seize and image devices. And they have a lot of these cases going on and so if something gets all the way to trial they may have to shift their focus to another case that they are currently working on. But the attorneys, the prosecutors may need more assistance getting ready for trial and that's where we try to step in. We've also developed our lab around a team-based approach to forensics so we have groups of three or four that work on our cases and so we can hopefully turn around these cases faster by splitting up the work among a few people who can work together with their areas of expertise. So one of the issues that we kind of have to combat is this idea that we're ever really completely done with digital forensics. There's not really a full forensics where you say, okay, we found everything that we could find, that's it. There's always more that we can do. There's always more that someone else, another analyst, could find if they had more time to really dig into things. So what are some challenges that we see when collecting digital evidence these days? First, as was mentioned, there are more devices out there. Earlier this spring we got a case into our lab that all of the equipment that was seized from this guy's home filled an entire one of our offices. And so that's way too much for us to look at. And so it's also not really critical for the entire case that we look at every single piece of equipment. The dusty computer that was found under the bed, that is a super low priority. But the question then comes is, okay, we have all of this equipment to look at. What do we look at first? What's the most important? That's where communication with investigators is really important because they have, you know, they knew this was the computer that was actually running, that was on the guy's desk. So you may want to start here. The second thing that we have to deal with is the increasing storage capacities of devices. There's a micro SD card that is small enough that it fits on your fingertip. Back in 2005, you could get 128 megabyte micro SD card. 11 years later, 256 gigabytes on something that fits on your fingertip. And so that's a 2,000 times increase in the capacity of something that's so small. As a comparison, what we say is that one gigabyte of data printed out on paper would be a stack of paper about 1,000 feet tall. So you have something this small that is 256 stacks of 1,000 feet paper. That's a lot of data to look through. Another issue that we have with increasing concern these days is encryption. And if a device is really well encrypted, you know, you have iPhones these days that come encrypted by default. You have the newer types of tablets like from Microsoft. If you have a surface tablet, there's a good chance that it comes with bit locker encryption enabled by default. So the users may not have even known that or set it up, but it comes that way. And so with good encryption products, it becomes very difficult to bypass that encryption. So if we're dealing with we don't have any information, here's an encrypted hard drive. It makes our jobs much more difficult and that can take, you know, months or years for it to actually complete. And encryption is being used not only in the devices but in communication as well. People are using messaging products and a lot of them are turning on their encryption. So that if you try to intercept the communication, well that's encrypted too. And so that makes it much more difficult to get access to the communications. Encryption also changes the way that we respond on the scene. So if we think that somebody might be using encryption on their computers, we don't want to shut the computer down right away because that just reverts it back to being fully encrypted and fully protected. If the computer is up and running, we want the first responders to take some different approaches to image RAM because that is where the encryption key would likely be stored in a running computer. We want to check for encryption to see are there encrypted drives that are mounted in an unprotected state on this system. And then we can act appropriately and create a forensic image while the system is up and running. So we get all of those decrypted files while the system is running. Another thing that we have to deal with is the accessibility of the storage as well. So it used to be that everyone had a desktop computer and you could pull off the lid and you could pull out the hard drive and connect it to a write block, which is what one of these two devices are. And that write block gets connected to our forensic workstation and it's great. It was really easy to copy all the data off of the hard drives. But these days devices are not made to be opened up by people, especially very easily. So you've got phones, you've got tablets, you've got computers, especially some Macs these days that are really difficult to get open. You need special tools. There's glue everywhere. There's special screws. And so it's not just as easy as opening it up and popping out the storage medium and then imaging things. And especially with cell phones, what we need to do is that cell phone needs to be running. It needs to be on, which is much different than our normal process of having a computer turned off and nothing's changing. But the cell phone has to be on so that the computer talks to it and it basically says dump all your data out for me. And so that means that our cell phone forensic tools have to be aware of lots of the different cell phone makes and models and features and security features that might be trying to protect data there. And there's a lot of different types of cell phones out there. The other thing that we need to keep thinking about is that we're finding all of the potential sources of evidence. Specifically, I'm thinking of syncing and syncing to the cloud. So we grab a computer, but the data might not actually be on the computer or the computer or the cell phone might be encrypted. And so we need to go somewhere else to get access to that data. So that might be in cloud storage somewhere or the computer might be syncing to the phone. The phone might be syncing to the computer. So if we can't access the data on one, it might be on another device as well. So that's a little bit about the collection of the data. Now in terms of the analysis that we do back in the lab, one of the challenges that we have is really understanding the request that is being made by the prosecutors or the investigators that want us to work on their cases. And it's important because it drives the rest of what we do. And so sometimes it's trying to figure out what does the customer actually need. They may ask us, can you find this IP address on this computer? We can and we may find it, but does that really help them? Our bigger question is what do you need to prove? Because then we have the expertise and knowledge of knowing where to look to find things that happened on the computer. So if the request is we want to find out if someone remotely logged into this computer, then I know that I can look in several different places that may include those IP addresses, but it may include other things that the prosecutor and the investigation team haven't really thought of. And so then once we actually understand the request, we want to make sure that we fulfill the request. There's millions of files on a computer, and so we can't just start looking around at things. We have to make a logical approach and make sure that at the end of the day, we can say the request was met. We have an answer for you in a clear language that they can understand. And we don't want to just find evil on a computer. We want some user attribution. We want to be able to say this is how this information got onto this computer. We might be even able to conclude who was using the computer or where the computer was at the time. This background information is very helpful to help understand what was going on at the computer at the time and how it relates to the rest of the case. Another one of the challenges we have is keeping up with technology as things change. There are about three million apps on the iOS app store and the Google Play store. And so the forensic tools that we have don't know about all three million of those apps. So it requires us to play with the technology to test things out, to try and test and learn and verify. And if we need to, be able to make our own tools to interpret some of the data. And so the final point that I'll leave with is that I should say specifically digital forensics is sort of a combination of science and art. So we have to have the background about the technology to know how it works. But then we also have to kind of have that investigative mindset to think about where the evidence could be and to think about how it relates to the rest of the facts of the case and how it relates to evidence in the real world. And then if we come upon a hit on a computer from like a keyword search, like this in the green box up here, we found through a keyword search on remote detonator. This was an attempted murder case. We find that and we see all this text in the green box and we're not really sure what that is, right? Except we're keeping up with technology and the research that is coming out and we're able to recognize the bits of information that are in there. Like the thing in the red box that series of numbers is actually a date that's been encoded. So then we were able to tell, oh, this is a Google Analytics cookie that was on a cell phone that was backed up to a computer that shows that even though the user cleared their Internet history, that they searched for homemade remote detonator, visited a website, and this happened the night before that they went and purchased the equipment from Walmart the next day. So it's finding out about these weird hits that you see on the computer and then kind of making sense of that data and being able to describe what it actually is and why it's meaningful. So being able to learn and test things is important, making your own solutions is important, and then of course at the end of the day being able to explain it to a non-technical audience, a judge, a jury, your prosecutor or your investigating team is very important as well. Let's see where we're at here. Might need a lifeline for this one. There you go. The other mark. So I just wanted to start by sharing a quote and I think this quote really captures what I plan to speak about today. Having worked in forensic science for 26 years, I believe the contributions that forensic scientists make to the criminal justice system are critically important. Their work improves our safety and ultimately our quality of life. When I started with the Pennsylvania State Police in 1991, violent crime was peaking in the United States. DNA was just beginning to be implemented on a wide scale basis in most state forensic laboratories and the FBI would soon pilot a national DNA database. Also a national ballistics database for gun related evidence would follow later that decade and advances in algorithms were improving the accuracy and search capabilities of our fingerprint databases. Did these advances in forensic science contribute to the decrease in violent crime? I'll leave that for the social scientists to debate, but there's no debate in the tremendous technological advances seen in forensic science over this time. The field is far from static even for techniques that have been used for decades. So on the other side of the equation, the innovation on the criminal side, we can look just at drug evidence. When I started and I did some drug analysis testing, our drugs had names like heroin, cocaine, the occasional PCP or LSD. Today we have synthetic cannabinoids, the synthetic opioids, and drugs with names like U47700. On a related note, I shared with Lindsay a seizure that happened in the state of Nebraska, fentanyl seizure. And with a lethal dose of two milligrams, this fentanyl seizure had the potential to kill 26 million people. One seizure. So just think about that. It also impacts how the forensic scientists deal with that type of evidence in the laboratory. On the innovation on our side, a DNA test of a known reference sample could take a month when I started working in the laboratory. Today with new rapid DNA technology, this could be done in about an hour at a police booking station. So without even a forensic scientist doing the work. So as we implement these new technologies, we must continue to strive for quality and excellence. So just a little bit of background. My office is a tap is within the criminal division of the Department of Justice and our sole focus is international law enforcement development. Essentially our mission is to strengthen our partners around the world so they are better able to address transnational issues such as terrorism, organized crime, trafficking of all kinds and human rights. Our work is in all areas of forensic science. So I won't focus on one discipline today such as digital evidence. Rather, I'll highlight our approach to ensuring that forensic evidence is reliable and should deserve the public's trust. So why is forensic science important to our mission? I believe the answer is most people to in this room and watching online. We use and share forensic evidence to investigate and solve any number of transnational crimes. To meet these challenges, we help countries implement new technologies such as techniques to analyze digital evidence or create a national DNA database. But I would also ask you to think about the indirect contributions that forensic science can make which are important to the rule of law. Does public confidence in convictions increase when forensic evidence supports other testimonial evidence? Does public trust in the rule of law improve because of non-bias scientific evidence? Are sexual assault survivors more likely to report a crime and believe they have access to justice because there is a forensic DNA laboratory in their country? I believe the answer is yes to these questions. Ensuring the exchange of reliable forensic information between countries including multi-jurisdictional use of evidence is a key reason why my office provides assistance in forensic science. As a result, we believe in the importance of international accreditation under the applicable standards issued by the International Organization for Standardization, or ISO. ISO accreditation allows a laboratory to demonstrate that it is conforming to international standards and best practices. Accreditation is also important for the sustainability of our government's investments. Best practices are incorporated into laboratory SOPs and the continuous improvement model provides a management framework that will build upon our efforts long after our assistance ends. The importance of accreditation in our work has been reinforced by our own domestic experience, which is filled with many examples of amazing applications of science to criminal investigations as well as some work that quite frankly misrepresented the science. Seeking to improve the practice of forensic science, Congress asked the Department of Justice to fund our National Academy of Sciences to do a review of forensic science in the United States. They produced their report in 2009. The report may have been more critical than some of the community expected, but it has been shaping our work ever since. The report had a number of recommendations, some only relevant to the Organization of Forensic Science in the United States, but others relevant to all forensic laboratories. A particular interest, the report recommended that ISO accreditation should be mandatory for all forensic laboratories. A related recommendation focused on what quality assurance systems should identify, namely mistakes, fraud, and bias. While the report recommended mandatory accreditation, many people in the field recognized that accreditation wasn't the final solution, rather a minimum level that all laboratories should attain. We are not alone in recognizing the importance of international accreditation. The European Union has stated the importance in several council framework decisions. The EU acknowledges that forensic evidence may be used in multiple jurisdictions and international accreditation should allow evidence produced in one country to be seen as equally as reliable as being produced in another country where it might be introduced in a trial. In fact, in a 2016 decision, they reaffirmed this position and focused on forensic evidence and databases where evidence that can be exchanged at an international level, such as information related to guns, drugs, and explosives. They went on to say, and I quote, beyond traditional forensics, recent events have highlighted the urgent need for a swift exchange of reliable digital forensic data that can be used as evidence in court beyond the jurisdiction of the member state of origin. Therefore, accreditation of forensics procedures should also be pursued as a matter of priority in this area. Ensuring the exchange of reliable forensic information is a significant reason why my office provides this type of assistance. We have had success working with countries to obtain this achievement. So far, nine of our partner countries have obtained ISO accreditation in at least one area. Many of the countries are using the same accrediting body that we use primarily in the United States, A and A B. Mexico is our largest program, and we obviously have shared security interests. In Mexico, I am proud to say that eight states in the federal attorney general's office have achieved ISO accreditation in at least one forensic discipline. The state of Puebla just became the first state in Mexico to obtain accreditation in our six focus areas of DNA, chemistry, ballistics, fingerprints, question documents, and crime scene investigation. With this success, I am often asked how we help facilitate these results, what is our model? First, we emphasize that we are peers in the international forensic community. We are not exporting American standards, rather these are the norms and the best practices of the international forensic community. Second, we try to simplify the process in our language, but not the rigor. For example, one of our advisors breaks down the accreditation process to 10 steps, which psychologically seems much easier to achieve than trying to meet the hundreds of individual ISO requirements. Lastly, we turn our partners into leaders. We have scientists visit partner labs that have already achieved accreditation to learn from their experience or to observe an internal audit. We also push our partners to assume leadership roles in the international community. Just last week I was joined as a keynote speaker by the laboratory director of Costa Rica and the quality assurance manager from Baja, California, Mexico at the annual awards luncheon of the American Society of Crime Laboratory Directors. Pushing our partners to be leaders and mentors simply reinforces the first point that these are the expectations of the international forensic science community and not just the United States. Accreditation under the ISO standards provides a framework for a high functioning quality system. However, accreditation is more than standard operating procedures, validation studies and chain of custody records. For excellence, a laboratory needs leaders that value the culture of quality. In fact, several of ISO's foundational management principles are related to the values of the organization and its leaders. In particular, ISO says that leadership should create a unity of purpose and direction for the organization. It also asks leaders to commit to continuous improvement as a permanent objective. In other words, ISO is asking for the organization to have a culture of quality. Why is an organizational culture of quality so important? It's important because in the end, this is still a human endeavor and as such, nothing will always go perfectly. Like many forensic professions, forensic science requires strong leaders. When potential issues come to light, which they eventually will, the leader may have a choice between addressing them immediately or hoping they go away. But as we have seen, ignoring issues ultimately leads to larger problems at the end. And given the consequences of our daily work, the public deserves forensic laboratories that strive for excellence. Thank you. All right. Now the other, Mark Grant's Secret Service. Thank you very much for the invitation today. I'm going to start out with a quick history lesson just to make sure I fulfill my contractual obligations to promote the agency. I do a lot of these presentations. Inevitably, there's always a handful of folks that are unaware that the Secret Service conducts financial investigations, let alone cyber investigations. So everybody probably recognizes this picture, Curry or Knives lithograph from April 1865. Interestingly enough, at the time, there was no national banking system. So the $1 bill from the bank in Washington, D.C. was markedly different than the $1 bill from the bank in Philadelphia. And what happened, what became affecting the economy was Conor Fit was running rampant. Again, if you don't know what a dollar bill looks like, it's difficult enough as it is today. But if you don't know what a dollar bill is going to look like, it's hard to tell what's real and what's not. Numbers at the time indicate that Conor Fit bills in circulation were close to 30% of the total currency in the United States. So obviously the president and the leaders recognized that that was an issue. So on April 14th, President Lincoln signed an order for the creation of an investigative arm under Treasury to create a group to specifically combat Conor Fitting and essentially secure the nation's financial infrastructure. April 14th, he also had a date with Mrs. Lincoln to go to Ford's Theater, and we all know how that ended up. But the ironic part is the day that we were created essentially, the day that President Lincoln was killed had nothing to do with us being involved in protection. Obviously that's what we're most well known for. That really didn't happen for about 40 more years when we kind of slowly took over that. But we have been doing financial crimes for 150 years. Incidentally, we were actually doing digital crimes before the internet. Before the internet, I believe it was ARPANET, and we've got a case going back to the early 80s where we were working with DOD resources and helping them out. So we've been doing it a long time. We're very good at it. So what I'm going to talk about now is I'm going to kind of elaborate on a couple of things that Joe and Mark talked about. The first one is accreditation. Again, Mark just talked extensively with that. When I started Secret Service in 2001, I was fortunate enough to be trained as a forensic examiner. And when I came back from training, I think Ovi Carroll was one of my instructors who's still in DOJ today. I had an office. They were nice enough to take the carpet out of the office and put the anti-static tile in, give me some extra power strips, a different lock on the door. But that was my lab. Obviously that doesn't necessarily meet today's accreditation standards. But backing up 20 years or so, kind of in the mid-70s, is when this ASCLAD certification kind of started to take hold. And the goal at the time was to create a framework. Again, as Mark discussed, to have a set of standards that can be applied across the board. The issue that we have today that I see specifically in my world is when ASCLAD was designed, as we talked about, it was for blood. It was for DNA. It was for ballistics. Fingerprints haven't changed in a long time. DNA hasn't changed for hundreds of thousands of years. So what you're testing is a constant. The technology changes as we talked about, you know, what used to take a month to do a DNA sample now takes hours. But what you're testing is still the same. The issue with digital evidence, though, is if you took a time machine and brought my lab back from 2001, it would essentially be useless today. Everybody's cell phone in here. I would have zero ability to look at that cell phone. Far less the ability to look at the smartwatches that everybody's wearing, the Alexa device that everybody has in the room, or even the vehicles that we drive. Vehicle forensics now is a growing industry. And even something as simple as your laptop. The hardware connections, the right blockers that we've talked about, all that stuff has changed over the years. Part of the accreditation issue is standardization. And again, when you're looking at DNA that hasn't changed in hundreds of thousands of years, you know what you're looking at. In the digital world, things change frequently. So it's the issue that the service has. And again, we agree with 95% of the accreditation argument. But our tools are changing literally daily. Again, I talked about my tools from 2001 not being applicable today. The tools that were assigned in 2017 oftentimes aren't applicable today. Example, we're working a case right now looking at some Romanians that are creating gas pump skimmers. Gas pump skimmers are kind of the scourge of the cyber crime world. They're tangible. We see them. We see them in the news. And it's a giant pain. The guy that we're looking at is building them in his basement. He's buying parts on eBay. He's grabbing some old cell phones that have, you know, the batteries have kind of died on. And he's cobbling something together to make a device. And he's making 10 or 20 at a time, shipping them over here and then they're being used. Every time we find new skimmers, it's a new skimmer. I can't go to the forensic store and buy that device. It's going to interrogate that piece of hardware because this is the first time we've seen it. And our issue with ASCLAD certification, with accreditation is the tools that you're using, letter of the law, and Mark and I were talking earlier, there's some interpretation that these are open to. But letter of the law is these tools have to be tested. They have to be rigorously tested to be verified. Sometimes it takes months, years to verify that a tool is going to provide the results that it should on a consistent basis. If we're getting a device, we're seeing it for the first time. I can't wait a year to do the forensics on that device. So it's something that we look at. Again, 95% of accreditation we feel very strongly with. We try to match as much as we can. Our forensics laboratories across the globe have a very strong track record. Again, I guess you could say fortunate enough to testify as an expert witness in federal court and had no issues with my display of the evidence. But it's something that there needs to be flexible. Again, we talked earlier about digital forensics being kind of a mixture of science and art. Not every piece of regulation can be applied to it. You do need some of that flexibility. The other thing that I want to talk about today is encryption and data privacy. Obviously, this is something that we could talk about all day long and is kind of exhaustingly discussed in the news on a daily basis. But just to touch on a few specific issues and how it applies to investigations. As investigators, we're all always playing catch up. Rarely do we see something, a new emerging technology, a new emerging opportunity for criminals. Think ahead and know what the criminals are going to do and have a plan, have a practice in place before that crime occurs. We're always one step behind. That's the lot of an investigator. We've come to accept that. But what we're concerned with now is encryption is getting two, three, four steps ahead of us. Increasingly, there's areas where we're completely dark. The level of encryption, the level of technology is advancing quicker than we can keep up to. And we're getting to the point where there potentially could be large swaths of evidence of data that are completely out of reach for law enforcement for the courts. Historically, there's always been privacy. Everybody in here can have a safe at home. Everybody can have a lock box at the bank. But even your conversations with an attorney, with your wife, with clergy, all those are protected, all those are private, but they're still within reach of the rule of law. And we're rapidly approaching a time where we just don't have that capability technologically anymore. Interestingly enough, last week I was at a conference in Las Vegas, and James Comey was the keynote speaker. Everybody remembers 2015, the San Bernardino shooting, and the iPhone was kind of the center of that controversy from a privacy and encryption standpoint. Initially, the FBI couldn't get into the iPhone. There was a very strong concern that there could have been additional members of this attack group. And so there was pressure put on Apple and eventually a court case that compelled Apple to provide a backdoor to this device. At the time, Mr. Comey came out, and he was very outspoken, and plainly stated that, yes, absolutely, the government should be able to compel private industry to create backdoors. The interesting thing, last week Mr. Comey has kind of softened on that stance extensively. And I agree with him that there has to be a balance. We can't completely let the government, and I represent the government, we can't completely let them into our lives and be able to monitor everything. But again, there has to be a balance where in the right situations with the right side of approvals, the government can step in and can look at your most private investigations. Private investigation, private information. As I mentioned before, though, Secret Service law enforcement is always playing catch up. And in this instance, in 2015, we were able to catch up. Private company out of Israel Celebrate was able to develop a tool that could get us into that iPhone. But at the same time, as soon as iPhone, as soon as Apple saw that, now they begin creating something that Celebrate's not going to be able to defeat. And again, we're in that constant cycle. And as long as we're in the ballpark, it's not a major concern, but I think we have to look closely at the potential of a time where we do fall significantly behind. And if we do, the question is going to be, does public opinion change? Right now, the obligation is on us. The obligation is on law enforcement, on the government to find that solution. The question that I have is if crime increases rampantly, if there's unfortunately significant terrorist events where we can kind of definitively say that encryption aided that crime syndicate. Encryption aided that terrorist activity. If that happens enough, potentially that court of public opinion shifts, and now there is an agreement that, yeah, maybe we do need to force Apple to force Microsoft to force Google to create a backdoor. Again, right now that onus is on us. Apple will argue that if they create that backdoor, that's essentially just creating a vulnerability which creates more weakness, which then makes the entire system weak. I would argue that if Apple creates that backdoor, if we look at a device, a digital device, just as if it's a structure, if you put a good lock on that door, if you reinforce that door, you put an alarm system on that door. Again, this is all my background from working on alarm systems at the White House. You can make that door every bit as secure as that solid block wall. The issue there is, again, the public sentiment, and from a business standpoint, if Apple were to secure that backdoor, that's a significant investment, and it's probably one that they're not going to make a profit on. So, again, that's a discussion that hopefully it never comes, but potentially in the future you look at those issues, and again, public support could sway that. The other thing that I would argue is, you know, right now everybody kind of has that concern about government peeking over your shoulder in your emails. The reality is most everyone probably in this room has already kind of opened the door for Google, Microsoft, Apple. Again, just a few weeks ago we saw the story with Cambridge Analytica. It's a little frustrating from my point of view, again, that you see the comments, you see the talk in the news about law enforcement and government being big brother, but at the same time everybody is allowing Google, allowing Facebook to see everything that they post, everywhere they're at, everything they talk about, all of their friends. And we've seen, again, within the last couple of weeks that potentially that information can be used to manipulate us, to manipulate public opinion. So, to me, that should be as big of a concern, if not more than government. And finally, when you look at the way the feelings are with encryption and with privacy, someone can be concerned that it can go too far one direction or another. Everybody is familiar with GDPR, coming out of Europe right now. I'm sure everybody's email boxes are full of privacy updates. Mine, too, a lot of accounts that I didn't even know that I had or had completely forgotten about. Obviously, that's a massive piece of legislation. Lindsay and I were talking earlier, apparently it's a boon for the legal community, because a whole group of lawyers have kind of sprouted up to manipulate, they're not manipulate, to understand GDPR. One of the things specifically that law enforcement is very interested in is GDPR's effect on ICANN. ICANN, I don't remember the exact acronym. It's the organization that handles internet naming conventions, the IP addresses, the domain names. Historically speaking, if you created a domain name for the US Institute of Peace, you had to register that domain with ICANN. You had to provide them a name, an address, a phone number, email address, all the contact information. And then ICANN would publish that in the who is directory. And that is an amazing tool for law enforcement. I've used it thousands of times. The current interpretation, though, of GDPR and primarily our folks in Europe, is that publishing that who is data is then a violation of that person who's registered that domain. And so there's a very good chance that who is may go dark. So now if I'm doing a phishing investigation and I've got a domain that someone registered and I'm looking to see, you know, where is that domain registered? Who registered it? Where can I send subpoenas to? It's again going to be a black hole. And it's even gone a step further. There's actually a lawsuit that was enacted on Friday last week. One of the online registers in Germany has the interpretation that not only are they prohibited from sharing that domain registration information, the contact information, they're not even going to collect it. So if you open up a domain through this German, you know, web service, they're not even going to ask you your name, your address. And again, from an investigative standpoint, this is something that really needs to be addressed. Unfortunately, do we have any ICANN lawyers in here? I know they're based up their own little ways. ICANN and honestly law enforcement is probably linked to the table at trying to lobby for changes in this. So just another indication of how rapidly things change in law enforcement, especially in the digital world. So I look forward to our discussion. And with that, I will turn it over. Thank you so much. So we still have about 40 minutes for discussion and questions. There's two microphones on either side. And I highly encourage asking questions so we can get a discussion going. But if you do have a question, just raise your hand. And there's two people helping who can bring you the microphones. But I'll just start with a question that's perhaps more aimed at the two of you. You both brought up the issue of encryption and that it makes your jobs much more difficult in the investigation and prosecution of crimes and also the new data protection laws that are getting stricter, particularly in Europe. But have you seen... So this encryption is being put in place as a protection for privacy of those individuals and to increase security of companies that hold your data. In the prevention of crime and cyber crime, identity theft, all of those issues, are these encryption technologies helping? Have you seen a decrease in the amount of crime themselves, even if it makes you harder to go after the criminals? Are there fewer criminals to go after because of it? You want me to start? Sure. All right, I'll jump on this one. Absolutely not fewer criminals. That's a growth industry. My area of focus for probably the bulk of the past 10 years or so has been Eastern European cyber crime. Not so much election interference, so no questions on that, but the Eastern Europeans specifically, obviously the Russian speakers, have been very focused on our financial infrastructure, credit card industry, banking industry. I see them effectively using encryption more than industry uses it. There's probably a good argument for that. For that single Russian hacker that's in his basement, in Akaterinburg, he's got one or two computers that's relatively easy to keep that secure. And he's, to some degree, behind the giant firewall that is Russia itself. Whereas an industry group, a large corporation in the United States or in any Western civilization, has a massive infrastructure. And that massive infrastructure has a lot of doors, a lot of windows, a lot of openings that takes significant effort to ensure that all of that is secured. One of the things that I see a lot, unfortunately, is organizations don't know everything that they have. There's a lot of, especially in the smaller groups, they're looking for that silver bullet. So something as simple as strong passwords, which is kind of a building block to encryption. If you have a fantastic encrypted system and you maintain the default password that was on there when that system was installed, it doesn't matter. And unfortunately, we see that kind of time and time again. So I would say that their adversaries are probably more effective at the use of encryption than customers. Just to follow up on that, because you said one of the issues is the companies don't even know all the data that they have. They're collecting so much they don't know what's in their possession. One of the things in my understanding of the GDPR is companies will have a much better idea of what's in their possession because they need an affirmative legal justification for everything they collect. So they'll be collecting less and they'll know what's there. So for just looking at these breaches like Equifax or Sony, these huge breaches, had they been compliant with the GDPR, would those have been less harmful? Is there anything of value that you see in that? You want that one, Joe? I'll try to offer a suggestion. So knowing what you have is certainly step one, but then protecting it sufficiently is another clear step beyond that. So it would be making sure that they have enough safeguards implemented to actually protect the data that they now know that they have. And that's in terms of the perimeter, but also detection and quickly responding and being aware of what's happening on your network as time goes by. Companies often see that compromises occur, have occurred a long time ago once they see that an intrusion has taken place, that someone hacking in has been around for a while having seen the data on the network. And so early detection on those sensitive areas is certainly important. If I could add one kind of thought to that. GDPR, I think maybe one of the advantages, again I'm not completely sold on it, but one of the advantages everyone is aware is there are significant fines that potentially could be levied against your organization if you aren't securing your data securely. I think the problem in the United States that we see oftentimes, and I kind of touched on it earlier, is if it doesn't lead the profit, it's not a priority for these companies. And you look at a lot of the large data breaches that occurred, it's front page news for a couple of days, the stock generally dips for a couple of days, sales dips for a week, two weeks. But unfortunately in two or three weeks there's another data breach. And now the heat's kind of off of that company. And now the shoppers come back and the stock price comes back up. And so it's a simple return on investment. Do we spend tons and tons of money on network security? My apologies. Thought I turned that off. So we spend tons and tons of money on network security that essentially is just money going out the door and doesn't lead the profit when if we do get breached the worst that happens is we take our beatings for a couple of weeks in the media and then everything goes back to normal. And I think a lot of companies, you know, when they're sitting around at board meetings, I know from friends that are in the financial industry, credit card industry, they look at the beginning of the year at their budget and they go, we're going to lose $100 million in fraud. And that's just the cost of doing business. I think, like I said, GDPR may be a bit of a game changer because now, especially the folks that are doing business with European customers are under the gun that, yeah, there could be sufficient ramifications that's going to hurt the bottom line if they don't comply with the rules. Great. Thank you. And on that kind of theme of thinking about privacy and data protection, you mentioned rapid DNA and some of these newer technologies. When we spoke, there was also something sequencing. Next generation sequencing. Next generation sequencing. Those were the two kind of new areas that you highlighted of importance. Obviously, it's kind of a cliche at this point to say the law lags behind the technology lawmakers have been very slow to legislate things in cyberspace and new technologies. So when you're coming up with these new technologies of rapid DNA and the next generation sequencing, you could describe those a little more of how you foresee them being used and are their considerations of the ethical implications where maybe the law hasn't caught up to the technology. Sure. First, I'll say that I've been pretty much a scientist at a desk for the last 12 years. So my work in the laboratory is a bit stale. But the trends, the first thing to appreciate when people hear the word DNA, they think of something that's very personal. The first thing to keep in mind is that the traditional DNA that is conducted in the forensic laboratory, we are looking at areas of the DNA that don't code for anything. That's why there's a lot of variation and that's why we can individualize someone. So we are not looking at things that we all have in common. So if we're just looking at eye color, do they have the DNA for brown hair? That doesn't help us individualize a person per se. However, but there's always ways to push the envelope. So the ethical debates come up from time to time. A few years ago it was familial searching. So let's just say you have a sexual assault case. The person that committed that crime is unknown. We have a DNA sample. We put it through the national DNA database. We don't get a hit. However, they match in every allele that we are looking at except one. And that would indicate that the person that committed the crime is probably a relative, maybe a relative of the person that is in the database. So what should an investigator or laboratory be able to do with that information? So people think of it as being very personal when you're talking about DNA. But then on the other side, prosecutors will say, this is really not that much different than getting a partial license plate. And so is it unethical if you have five digits of a license plate to look at all the similar cars with that license plate and to be able to hone in on the person that committed the hit and run, if you will. So there is a number of things. But there's also, because we touched on this as far as some of the things on the genealogy and that aspect, this type of technology with the next generation sequencing will have more access to do this type of testing on a regular basis in the forensic DNA laboratory where we haven't really focused on it because it wasn't aiding us in the identification of an individual. With the new technology, you'll essentially be able to do both. But this has been used in less controversial areas, the remains that are found in a grave. People have done facial reconstruction to be able to identify the person, but they have been reluctant to add any features and interpret that. Having some of the ancestry markers has helped them provide a little bit more information and really identify the person that has been in that grave. So there's a number of different angles that it could potentially go down, but at least in the forensic scientists, we are always just constrained and working by what our legal teams and the investigators allow us to do. Does anyone have any questions? So the DNA conversation makes me think of some of the other applications as well with the interplay of basically private law. We're talking about regulations, we're talking about the EU, et cetera, but we're also talking about, especially with mobile telephony, we're talking about a lot of apps that have a very long pseudo contract between us and the company that we all click on without being able to negotiate. That is seen as binding law. That's something that was certainly in the DNA match for the Golden State Killer recently was pointed to by the company saying, well, there's one word in there saying other uses, and so therefore you would agree to this, so there's not any privacy issues, as opposed to there wasn't something specifically saying this will be available for this type of law enforcement use, et cetera. And so I'm wondering whether it's everything from what we saw with law enforcement agencies basically having to sign away their right or their ability to even tell the prosecutors where they were getting evidence, when they were using Stingray a few years ago, to user agreements, whether it's in apps or in telephony, how that plays into what we assume is the rule of law, because these are seen as binding legal contracts, but they're not necessarily coming into what we're used to thinking about, because there's cops, prosecutors, judges, and open courts where we can see rule of law happening. Instead, there's this other layer of law that maybe isn't as transparent as our assumptions when we're talking about rule of law in general. One of my thoughts specifically with that and a lot of the other topics that we talked about today is, unfortunately, my vision of, especially American society is we have a very short attention span. A month ago, Cambridge Analytica was the hot topic, and companies left and right were ditching Facebook, and people were canceling their Facebook accounts because of some of the exact same things you talked about. When you sign up for the Facebook account, it's an understanding that you're agreeing to their terms and conditions. You've got to click the box. You've got to accept. Obviously, nobody reads that 20-page document and looks for that other uses clause that's in there. But again, unless there is a consistent messaging, consistent stories, I don't think there's going to be enough of an uprising among the public that's going to make a whole lot of changes. Again, if we string a handful of those stories together, Cambridge Analytica, the Golden State killer is kind of a mixed bag because I'm sure most people in here that have done the DNA testing never imagined that it could be used for that. So on one hand, they like it that we caught a serial killer, but on the other hand, do I really want my DNA in a database that law enforcement is going to be able to look at? But again, my opinion is unless we see consistent issues, consistent coverage, I don't expect much of a change. Hi. I'm Jim McMillan with the National Center for State Courts. One very short comment and then a question which I hope will lead to some interesting conversation. One thing you have to understand about civil law countries, because this is where GDPR is coming in from EU, is that they are constrained by the whole concept of civil law, which is you can't do it unless we tell you you can. So this is why you were seeing the who is reactions and things happening from the European countries because they're waiting for guidance. They're waiting for the rules to be passed by the EU to give them guidance to tell them it's okay. You can do it. So I think that this will all sort itself out here in short order, but I'm sure that our friends at Europol and Interpol will be giving them some encouragement in those areas. My other question, my actual question for you is that I'm really fascinated by the whole idea of what's the digital evidence package that's now being done to put together by the prosecutor for the courts to prosecute the people, but also where do you think that this digital evidence package is going? How are we going to be working with it and that sort of thing? So I'm looking maybe toward our friend from the Department of Justice, but I'm sure that there may be some thoughts from all the panelists, so thank you. So with so much data available on computers these days, there's only a piece of it that is relevant to making the case and you'll quickly overwhelm a jury and judge and all of that if you offer too much. And so if we can minimize it down to what actually makes the point, then I think that would give the best chance of removing distractions from other stuff that just gets in the way. One of the things that I've observed with my squad, I've got a handful of forensic examiners. I had one individual, absolutely brilliant, but he had a little bit of a disconnect in conveying that information. I've sat in on grand juries on numerous occasions and it's particularly difficult when you're talking these cyber cases because you kind of have a mixed audience, hopefully they're awake, not always the case, but you're trying to explain IP addresses and that attribution. Unfortunately, I've even had the issue with some of my U.S. attorneys where you're trying to tell them that, yes, this guy, this Ivan is the guy sitting behind a keyboard in Russia and to draw that line from this credit card that was used at Walmart to buy a stolen TV all the way back to Ivan over in Russia is difficult. So that final report, to me, one of the things that I emphasize with my young forensic examiners is not only do you have to be accurate, but you have to be presenting that information to whatever your audience is. If you're talking to the CISO at the company, maybe he's very tech-savvy and he understands, but again, if you're sitting down in front of a judge or you're sitting down in front of a grand jury or jury, you need to be able to present that in a way that they can comprehend. And then reports are useful for the detail, but then anything that we can do to help visualize what's going on, if there are a sequence of events that we can put into a timeline that people can understand by looking at it rather than reading a bunch of dates and times throughout a report, that might give them a better overview sense of what's going on. So I've had a trial within the past year where there were a bunch of network logs and remote logins and you're seeing these router logs with these confusing commands, what it essentially is when we boil down to it is someone remotely logged into this router from some location and then they switch to a different user account and they logged into a different router. So if we can create a little map that shows the entry point and then the access of another system on the network, then that's a lot easier to visualize than just looking at a bunch of text-based logs with weird commands and terminology in them. My name is Shabnam. I'm with the Syria Justice and Accountability Center. We do human rights monitoring in relation to the Syrian conflict. So with social media, there's so much information and potential crimes that are being posted online that it's maybe hard to do user attribution for. But there's also so many NGOs and civil society groups that have cropped up that are focused on doing investigations and this is kind of broadening the scope of who's an investigator. And also with just normal people, individuals like you saw in Charlottesville, it was crowdsourced information that led to arrests of some of the perpetrators. What's the role of law enforcement in working with civil society groups or working with the public now that it's kind of diversified, who has access to information and who can look for information online that can lead to arrests? I'll kind of jump in with social media. One of the things that we've seen in explosion, obviously, you know, Twitter, Facebook, all that stuff, it affects us directly with our protective mission. Back 20 years ago and then beyond that, if you wanted to threaten the president, you had to sit down and write a letter and send it to 1600 Pennsylvania Avenue. The Secret Service little bragging right here has the largest ink library in the world because that's how we did forensics back in the day. When we got a letter that was threatening the president, we did, you know, we took it to our forensic lab, they did the analysis on the ink and were able to kind of narrow it down to a geography or a specific company that distributed that ink. That was all fine and dandy. Then the explosion and the internet happens and now it's as simple as a drunken tweet and now there is just an overwhelming amount. Just like every other agency, you've kind of got to cut through the chaff and find what's real and what's just people venting. The difficult prospect is if you don't jump and make the right conclusions and then somebody that we thought was maybe just kind of running their mouth and pushing the border of free speech and now two weeks later, a month later, they do something, then it's, you know, why wasn't that person under surveillance? Why wasn't that person arrested? Why wasn't that person, you know, had a more thorough job? Not so much social media. We had an individual that showed up at the White House and he had a hatchet in his belt. Clearly a mental health consumer, but he didn't make any threats. He didn't cause any problems. He didn't threaten the president. He didn't threaten himself. It wasn't a danger. So we're kind of hand strong right there. We can't do much with him other than tell him that the president's not going to see him today and you need to move along. A month later, he jumps the fence and then days later, our directors up on Capitol Hill asking us if you knew about this guy a month ago, why weren't, you know, why wasn't he under surveillance? If anybody's been to Lafayette Park, the vast majority of the folks in Lafayette Park, again, unfortunately, are mental health consumers. And we just don't have the resources to follow every single one of them. So it's very similar that, you know, and again, maybe AI, obviously AI is kind of the cutting edge, one of the big topics, and it applies to presidential threats. Hopefully it can apply to school shootings. I know we see that a lot where it seems like inevitably when somebody shoots a school up, we look at their Facebook accounts and there are some indicators. So I wish I had a better answer for you. I wish we did a better job at it. But again, that's something that is on the horizon. And again, maybe, you know, maybe AI is the solution because obviously Facebook's not doing a fantastic job or putting enough assets towards it as we've seen in the past. And one thing just to respond to that, actually, the Human Rights Center at UC Berkeley is spearheading an effort to develop an international protocol on open source investigations to begin to kind of set minimum standards in that field where civil society is working, as well as professional law enforcement and how to effectively use that digital online open source information in investigations, which is actually focused very much on war crimes, crimes against humanity and what's happening in Syria. One of the things though, we're trying to balance between setting standards and the speed at which the field is kind of developing and new tools are coming out. You know, right now we use YouTube, Facebook and Twitter as the examples, but in a few years those names might be obsolete, you know, if we put, if we use tools. So we're trying to focus on specific principles. But that kind of brings me back to this issue of setting standards and the quality control aspect and how you balance that with the fact that you have new tools coming out every day and not necessarily the luxury and time of testing them to the degree that you want before a case goes to court. So how do you find that balance? Sure, I'll start with that. So, you know, I think digital evidence is the extreme, but, you know, we focus on the ISO standards and generally these are developed for essentially international trade. So that a company that is buying a product or apart from a company in another country that they can ensure the reliability of, you know, what they are buying since everything is, you know, depends on each other when you're building a car or an airplane. But what we always say in forensic science, you know, it's one thing in an industrial setting, give you the example of the inks, you're making pen, that pen manufacturer can control all their inputs. In a forensic laboratory, we don't control any of our inputs. So whether it's digital evidence or whether it's DNA from a crime scene that's on a dirty rag and perhaps it's a mixture of five different individuals. So having that appreciation for the flexibility and again, this is where even in the traditional forensics, you know, I don't necessarily say art, but I say that the person has to be creative in their use and application of science. So all these, you know, things that come up using the ancestry websites to be able to solve a cold case, you know, a serial killer. I mean, that takes some creativity, whether it's on the forensic sciences part or on the investigators part. But having that standards, I think it really comes and it comes to light and what we've seen with the problems with forensic science in the United States is when you come to the trial, there's getting the investigative leads and we're always concerned that in the forensic laboratories our customers are the courts and the investigators and we want to be timely with our results. But when it comes to court, we have to be very precise in our language both so that we convey very complex topics and scientific topics in a way that a jury can understand, but then also that we are not misleading them in what we have to say and the significance of our findings. And that's where we've had some of the challenges. And that is part of the accreditation process. We labs do monitoring of testimony to make sure that the scientists are not misrepresenting their conclusions. But then there's also the Department of Justice is enjoying some uniform reporting and testimony guidance that they're pushing out. Now, from a federal level, the state and local forensic laboratories aren't necessarily bound by that, but it's a pretty good guidance. I think I saw a question up there. Just a follow-up to the comment you just made. How do you do uniform standards for presenting evidence for another country? In most cases, it will be a bench trial. There are very few countries that have juries. But how does ESA TAP work with that? Yeah, that's a challenge. Many labs don't have true admissibility standards for expert testimony. But we provide training in that. And what do we try to reinforce? Essentially what we said. I mean, there are many different professional organizations have code of ethics. And just going back to my experience when I've testified in court and what I share with other younger forensic scientists is that it is not our job to make the prosecutions case. And we have to accept that the defense can make points. But we have to really just focus on remaining objective in the work we do and actually to be true scientists. One of the things that the National Academy of Sciences report that I mentioned, one of their recommendations was that forensic laboratories should be independent of law enforcement and prosecution. That is probably easier said than done. If you're starting a forensic laboratory in a country where you haven't had one, that's probably wise advice. But in our situation and with the history that we have, it's probably not going to happen anytime soon. However, those independent standards and the accreditation process I think forces you to be much more conscious that you are really an independent scientist and not only working for the prosecution. We're trying to help find the truth like others in the court system. Well, I might move this to Joe in talking about the independence and not specifically working necessarily for the prosecutor's theory of the case. When you have this huge volume of data, so many devices, so many places to look and so much content being collected that you can't possibly look at at all, how do you make sure that your investigation is independent and that you're catching potentially exonerating information as well as what's incriminating, and then how do you convey that to the judges? Sure, so certainly as we're conducting our search with our tools, it lets us search across or view lots of data pretty concisely. So if we are looking for programs installed on a computer to see if one was used recently, then it would be easy enough to see the other programs as well and if there's one that proved innocence, then that's something that we could definitely make note of. Or if there's a search term that we run against all the data on the hard drive, we would get results back really quickly and if there was data in there that was exculpatory in some way, we could certainly include that in our analysis, in our report. The other thing that we deal with is defense experts and so they will look at the same data and they will be able to provide their own analysis and find points that help their case and so we are able to review those and respond to those and even participate in trials if needed to respond to the defense's claims as well. Okay, I think we have time for another question. Anyone else? Questions? Okay, well I guess are there any other points you want to make just as a final statement on what people should be focused on looking out for. I know artificial intelligence was mentioned as a few times is kind of one of the newest topics. But any final remarks? I would just ask, I know in our experience in testifying in court what we have to deal with is what's called the CSI effect. It's basically that the jurors have sometimes an unrealistic expectation of what can be done in the laboratory and really what needs to be done. Sometimes they want and demand that an unreasonable amount of work can be done to prove something that everyone has already agreed to has happened. So that is a challenge for us when we testify. I'm not sure if that's come to play in the digital evidence realm yet or not. It probably has. Enhance this picture. My bosses certainly have the CSI effect. When we're given evidence it's like why can't you find what you're looking for? I think from a broader picture, and I touched on it earlier, about the constant changing focus. I mean that would be my push we're talking earlier. I get invited probably once a month or so to come to presentations like this. I enjoy coming out. I enjoy talking to folks about current trends in cyber crime. It's kind of the generic title that I often use. And I hope that it continues to grow, to gain legs, to be a point of focus so that people in positions of power far above me make significant changes to allow for better freedom, for better transparency for folks. Again, a lot of that's driven by policymakers in this room, policymakers up on Capitol Hill. But I certainly don't mind coming out and talking about it. And I'm frustrated, unfortunately, more times than not, that we aren't doing more, we aren't doing better. But hopefully the continued discussion leads for an improvement over time. I think the importance of providing education to all of those involved is something that we need to continue to work on. Our lab educates prosecutors and other attorneys and judges on digital evidence. And so they're aware of what's capable and what's possible, what is stored on a computer, what's stored in the cloud so that they have some idea when they get a case that they're working on or hearing that they know kind of what's reasonable and what's possible and what to expect. So that kind of thing is really important. Great. Well, thank you so much. Let's give a round of applause to the panelists. Let me also thank Lindsay for doing this appropriate job in moderating. This concludes the first panel of our discussion today. Next, we'll move to a networking break and also engagement with a variety of different actors in this field and you'll actually have the opportunity. In order to effectuate that, we need everybody to actually go up the stairs in order to go to the main concourse where the platform has been set up and, of course, to aid in your adventure. There's coffee and refreshments. The other thing I want to mention because we are obliged like everybody else to both monitor and account as well as being low tech is that in your folders you will see an evaluation form that's in bright yellow and we ask that those participants, individuals here, please take a moment throughout the course of the day especially if you're leaving to take a moment to fill out the evaluations which are substantively very helpful, they gauge our interest. And then finally, remember that in addition to coffee and engagement with the host of actors, again, the list is in the materials provided to you. We will proceed immediately to the keynote lunch where lunch will be served and then we'll have the opportunity to engage in another heady discussion. So please continue and we look forward to it. Thank you very much. And please continue to enjoy your lunches. Also please indulge in the food. It was intended for the purpose of being consumed so don't be bashful. In thinking about the topic of the event and in thinking about discourses about the rule of law we're always at a point in which we're discussing both the rule of law and we're also talking about the contours of the subject which itself can be quite challenging. And so raising the bar and getting or having someone who is as qualified and well respected as Ambassador Dabaca is really our treat. His extensive biography, again, is in the materials along with all the speakers but I want to predicate this discussion with a very long and storied career that began in the fight for civil rights and has taken him to a unique position in the Obama Administration on one of the perhaps perennial challenges that we face both here and abroad and his expertise has looked at a wide variety of different areas that touch on today's subject. So Ambassador Luis Dabaca is the former ambassador at large to monitor and combat trafficking in persons and former director of the Department of Justice's Office of Sex Traders, sentencing, monitoring and as you can imagine just with the titles however extensive they might be, the responsibilities are even greater. So in that regard, I think we all owe Ambassador Dabaca first a round of applause for his willingness to come and speak here today but also I invite all the participants to please listen in on the sort of challenges and experiences that he can bring to bear in a very fruitful discussion and so with that Ambassador, the floor is yours. I love how the water matches the podium. So the last time that I was here for lunch at the U.S. Institute of Peace, the keynote speaker, Ambassador Paula Dobryansky, who is a giant of the State Department and a giant of democracy building, was Undersecretary of State and a lot of other important things, was standing basically here at the podium and we were all out watching her talk and a derecho storm came barreling into town and she didn't notice it and we were all horrified because we saw what looked like tornadoes starting to form out over the river. So we should probably have a signal if you see a huge storm, just raise your hand and let me know and we could all jump in the basement or something. All kidding aside, I want to thank U.S. Institute of Peace for this wonderful place to learn and to come together but what U.S. Institute of Peace represents for us who have worked as diplomats in the United States service. The idea that peace is something that is a goal in and of itself, not simply that peace is the time that you rest and reload between the exciting times in diplomacy but rather that peace is what needs to drive us and that's why I think it's very appropriate to talk about rule of law when we're here at the U.S. Institute of Peace. I'll talk a little bit today about rule of law kind of as a concept but I definitely want to thank the South Carolina rule of law collaborative as well as the former colleagues from the Bureau of International Narcotics and Law Enforcement at the State Department for the JustTrack program itself which we're of course all here as part of and JustTrack is not simply kind of skills training. JustTrack is not simply programs here in the United States or around the world but to me it's very much trying to take this ethos of rule of law and put it into action and so I'm going to talk a little bit about what that ethos actually is. But first of all and I guess it's probably four hours into a rule of law conference it'll seem a little strange but let's talk about what rule of law is. Now Vince Lombardi every year at the first day of practice would raise a football up in the air and all of these professional football players at the top of their game had played ball for their entire life he would then say gentlemen this is a football and I think that that's one of the things that we need to do we always need to start with first principles. What is rule of law? Let's think about that a little bit before we think about innovation and before we think about technology to advance the rule of law. Now a simple definition is that the rule of law is the restriction of the arbitrary exercise of power by subordinating it to well-defined and established laws. A little more granular the folks from the World Justice Project break it down into four pillars. Accountability, that the government as well as private actors are accountable under the law. Just laws, laws are clear, publicized, stable, applied evenly and that they protect fundamental rights including the security of persons and property and certain core human rights. Third, open government. The processes by which the laws are enacted, administered and enforced are accessible, fair and efficient. And fourth, accessible and impartial dispute resolution. That it's delivered timely by competent, ethical and independent representatives. Neutrals who are accessible, have adequate resources and reflect the makeup of the communities that they serve. There's another way of thinking about rule of law and it's something that we often do. And that is as a practice area, an infrastructure, sometimes as a business, funding opportunities, grant management and oversight, maybe even as a buzzword that comes up in development projects and foreign assistance and international meetings. And you can see that in how one well-respected and active non-profit institution describes their rule of law project. And I quote, we seek to strengthen legal institutions to support legal professionals, to foster respect for human rights and to advance public understanding of the law and citizen rights. In collaboration with our in-country partners including government ministries, judges, lawyers, bar associations, law schools, court administrators, legislatures and civil society organizations, we design programs to respond to local needs and to prioritize sustainable solutions to pressing rule of law challenges. We employ rigorous and innovative monitoring and evaluation approaches in assessing the quality and effectiveness of our programs. Not exactly the stirring rule of law challenge that gets us to the barricades, but sufficient for a grant application. I'm going to use my speaker's prerogative instead to kind of mash all of these concepts together into a definition of rule of law that you should not follow. You should follow what the folks from University of South Carolina tell you. Thank you for coming to this session. I heard you had a very good morning session and lunch speaker, so we hope to keep this after lunch session as exciting to keep you all engaged. So this is a panel discussion and we're really going to be talking about how technology, specifically mobile and social media and what kind of impact it's had on the rule of law. So if we could maybe have my slides put up here. Actually, okay, this is an older version, but that's fine, we'll go with this. So the official title is Innovations in Engagement, How to Social Media and Mobile Apps Impact the Rule of Law. So I'm Rohini Shrihari and I'm the Chief Data Scientist here at PeaceTech Lab. And for those of you who are wondering what PeaceTech Lab is, we are actually right here within the USIP and our group focuses on data, technology and media for peace building. So I direct all the data programs and we're working on a very interesting platform. Maybe I'll briefly speak about it. But if we could just go to the next slide for a second. Pardon? Oh, okay. Which one is it? This one here. Okay, great. I'll try again. This big green button here. Okay, could we have the next slide please? Thank you. Okay. So what I thought I'd do is just sort of briefly introduce the topic and give you some examples which are really interesting I thought in terms of what has been already accomplished. But most of it is going to be a discussion with some great panelists, some of them who I just met for the first time today and I'll introduce them to you in a few minutes. So the way we're going to run the panel is I'll give a brief intro no more than 10 minutes on the topic. Mostly some examples. And then we're going to have each of the panelists introduce themselves and give some opening remarks. And that will be about 30 minutes. And then after that we're going to have a discussion session where I get to ask questions and they will respond. And then after that the audience has an opportunity to also ask questions. So I think there's plenty of time for discussion. We want this to be as interactive as possible. So with that we'll get started. So if we could have the next slide. I just wanted to give some examples of different kinds of technology that's been used in this area of rule of law. Press this one right. Really hard. Okay. I think I'm going to have to start with the Me Too movement. It's been incredible how fast and how rapidly this has taken hold of our society. I think in a period of two weeks there was something like 3 million references to this on Twitter and social media is really what caused this to really explode. In the media and discussions we only hear about the celebrities and the big cases. But I think many would agree that the real impact of this is not on the celebrities but other people, other women who really have now feel empowered to come forward and express what they've been experiencing. So I decided to showcase this as an example of really social media and how it's making an impact in terms of justice. So some other examples. Okay. There we go. Hate speech. There's been a lot of work on hate speech. And this is part by the European Union where people can report on hate speech and hate speech we've shown at PeaceTech, in fact we've been working on hate speech quite a bit also in different areas. Mandola is focused mostly on the European Union. Our work has focused on South Sudan, Kenya, South Africa, primarily the African continent. And the important thing really is that we've shown actually that a rise in hate speech can be directly correlated with incidents of violence in some of these countries also. So this is an important type of initiative and I know that Facebook is also doing quite a bit in this area. Just moving forward this is one of the earlier efforts that many people were probably familiar with. I paid a bribe where citizens can actually report and provide analytics and so on. It's a good initiative but in the discussion session I think we're going to come back to some of these sites and really talk about have these really been effective? Have they made an impact? Do people feel empowered to report? Are they worried about reporting these kinds of things? There are a lot of issues here but nevertheless it's a great example of social media being used to report in this case corruption. And unfortunately the corruption seems to continue despite these efforts. Hafez this was a mobile app that emerged during the 2018 Iranian protests on our site so people can send news about human rights violations but the mobile app also provides really useful information to people in terms of where to go for help if they're arrested or if they are persecuted. Names of attorneys, things like that something that is really caught on this particular one, fairly recent one. I respond with digital identity all about how people including people in refugee camps can be provided digital identities and through technologies like blockchain and so on which would then enable them to get help and the kind of assistance that they need. Just in case anyone's going to be here tomorrow there's a major conference tomorrow being held at USIP Blockchain and Social Impact so if anyone's here and would like to attend I think that's also kind of interesting. This one I'm not going to talk much about because we're very fortunate to have the founder and CEO of this platform right here with us today who will talk more about this IKMEC so this I wanted to highlight because this is an organization a company actually that we run an incubator here called the PeaceTech Accelerator which is an incubator for companies both non-profits and for-profits who are doing something in terms of making a social impact globally and it's actually for those of you who have been in this building it's in the basement of this building which we affectionately call the Bat Cave it's a pretty cool space actually and about four times a year we have cohorts of companies, seven, eight companies from all over the world who come and spend eight weeks here and they are mentored in terms of business technology, marketing, financing all of that with the accelerator called ICMEC but basically they work on it's the center for missing and exploited children and when they came here one of the things that we were able to do was introduce them to folks at Amazon Web Services who have technology for identifying faces in crowds and people have seen maybe not so good uses of this but in this case they're able to pick out children in these photographs and match them against reporting and we had a really success case a few weeks ago when someone actually came forward and said I believe that I was kidnapped 17 years ago or something so it was amazing so this is a really nice example of how we're being used for good so I think with that I'm going to sort of stop talking there's a lot of challenges some of them I've sort of briefly mentioned in terms of the data there's issues of how you collect the data privacy of data, regulations and most of all impact of all of these types so we're going to turn it over to our first speaker who's Jeffrey Arresti and he's the president of internetbar.org so Jeffrey, over to you thank you and I'll just look for the slides we have the video queued up by any chance there's a video, a link that we wanted to run this is the work that we do we go to a place like Timor-Lest and this is the work of an international human rights lawyer, David Levy who was able to document the transition of Meli Fernandez from a child fighter who was in the hills there was no respect for the rule of law there was no respect for the government Timor-Lest, there was no and yet through music we were able to at least start the process of creating trust and identity and give a group of people the chance to put down their arms and build a culture of respect for the rule of law which in a place where there is no law is the first step that you have to take in getting started Meli has turned not only into an artist but also an incubator of change for social justice in his own community but just a little bit of background about myself I'm an international business lawyer for 40 years that means that I go back to the time when the PC first came out the TRS-80 first came out the first Mac came out and I had all of them from the very beginning when I saw the first killer app I was working in an accounting firm the PC-1 came out along with an electronic spreadsheet called VisiCalc and literally hundreds of computers arrived to Coopers and Librand within the course of three or four months with auditors running around the offices giving each other high fives saying you just press the Recalc button and that would save them hours and hours of painstaking pencil-like work so I was very interested and got active I was in Massachusetts as a lawyer for all these years and growing up in that environment and I was one of the founders of the American Bar Association's tech show so I can speak confidently by saying that in the years I've practiced there has not been a killer app for the legal profession in fact law is the reason why technology can't innovate law basically stops innovation because of the jurisdictional nature of what is going on and so when you start thinking about it you ask yourself the question well if several times this morning we hear that technology is so fast and law can't keep up and yet we keep throwing technology at it aren't we missing half the equation probably the most important half how we make laws and how we enforce laws is what needs to change and that's pretty much what I would like to talk to you about my background as a lawyer though began representing my dad in the shoe business so back in the 70's and 80's he traveled all over the world and with letters of credit and telexs he basically was doing e-commerce we just now have e-commerce democratized in that kind of a setting but all that same type of technology has been available for years and we've been able to do it there's a professor of law at USC Jillian Hadfield who wrote a book called Rules of Law for a Flat World and what she essentially said in that kind of environment is that in order for there to be a new way to make laws we need to rely much more heavily on the private sector and mobile phones gives us that ability well I'm going to just click through to get all the points up on the slide if we can mobile phones essentially are in the position of connecting all of us with each other around the world this sense of hyper-connectedness that exists in the world as spawn revolution so if there's any question about the potential for making rules of law change with this hyper-connectedness you just have to take a look at the revolutions now they created voids and in the absence of law in those settings unintended consequences could occur so there needs to be some thoughtful process as we work forward now I work with mostly millennials in a small organization you'll see a couple pictures of the people that I work with a little bit later essentially millennials want to radically collaborate with each other they do not want to have silos that bind them in any way to each other they want to just connect regardless, aim at a particular solution and then make it happen in real time regardless of the platform that they're using and so when law becomes an issue in that kind of setting it's because of the paradox of reform most of the conversation about reform at the institutional level when you're talking about government and reforming law only affects a handful of people the enfranchised the disenfranchised can only reference societal norms and then they have to make change from the bottom up if we want to engage people like Mellie in the context of making changes in laws we have to give them that kind of access and then come up now you've heard of course all about the top down efforts which exist at the international diplomacy level at the public international law level but if you think about bottom up efforts from the ground up the most important things that I will have or at the bottom of this slide which are mobile phones self sovereign identity blockchain smart contracts collaborative environment we have companies that are ready to work on all of these areas right from the beginning and all we have to do is put a deal and put them all together so let me give you the names of companies that you'll see that we're already partnering with to get this job done the world justice project was actually the first seed funder of the internet bar and giving us the opportunity to focus in on how arts and culture could become a foundation for reaching normative behavior in a society and to use music as a platform to express that so that even though music is not the place where you typically think of as a lawmaking society it happens to be universal language and of course if you grew up in the 60s you know that music was really the foundation of a major revolution of thinking in this country so societal norms certainly start in that kind of a setting but now if you're in a place like Timor-Lest or any other of the developing countries that we're in and we'll talk a little bit more about that you're going to places where there are people that have no identities half the world three and a half billion people do not have any kind of an identity so as we talk about digital identities and all the problems that you're facing with digital identities the choice to make of working in the developing world where the justice system is not an obstacle to change is a much not only is a smarter choice just from a legal development standpoint but it's a smarter choice just because it's the right choice to make from a social justice standpoint and so we work in those kinds of environments self-sovereign identity which is a global public utility that's been created by a group out of called sovereign in the global public utility setting there are other groups that have also adopted the self-sovereign identity model is a model which essentially reverses the whole concept of identity from the ownership that we've all given up when we clicked I accept to every terms of service for every social media platform and every email service and every place that we've gone we're starting with self-sovereign identity in the context of we each have to make our own claim about ourselves think about it in the context of the Syrian nurse who's worked for 25 years now finds herself in lesbos and the public international legal framework for refugees and the national laws do not give her a chance to ever be a nurse again not only is that a travesty for her personally it's a travesty for the world as a whole because she has a lot of skills that should not be stopped because of the ravages of war yet how can she in an environment of a refugee camp establish her identity the self-sovereign identity approach has the notion of making verifiable claims the term verifiable is a term that every lawyer should grab onto if you want to see where the future the practice of law is going to be that term verifiable is where you're going to find it because as she is there next to somebody who has to receive information before it gets put up into this immutable blockchain that's not going to change ever how do you know that she's telling the truth that she worked in this hospital doing this kind of work for the past 25 years she's going to bring some people around her she's going to give herself some indications of support of people that can verify what she did but that's only going to be a starting point and then maybe if the camp in some way she'll be able to get a chance to work and maybe a doctor from the developed world doctors beyond borders comes down and a LinkedIn kind of environment starts giving her check marks for work that she's doing and little by little she might be able to build up attributes of identity which she herself controls because the way the blockchain works nobody else will have access to any of those attributes except for her so self-sovereign identity is an aspect and evidence sovereign.org has made that open source platform available to all of us internet bar the organization I founded with several others in 2005 we are the folks working on the identity for all best practices concepts of best practices of course would be focusing in on that nurse how do we establish identity and verifiable claims in that context how do those best practices applying different environments that might be other types of camps orphanages other countries around the world eventually they have to make the way to some international standard and we're talking to the international standards groups from the very beginning to make sure that that works now if you have an identity you then also want to have an ability to use that identity and if you have to demonstrate it we partnered with a group called odem.io which is an incredible group that has already built out ways to take these claims that people make about themselves and turn them into certifications so those certifications could exist in lieu of diplomas or other forms of certifications and then once you meet certain tests and standards that can also go up in the blockchain they partner with us that as we go from camp to camp or orphanage or different country environment they're going to be different paths of certification that we're going to have to take before something goes up on the blockchain another group that we work with is Vipo now this is interesting because if you have an identity what are you going to do with it that fellow Melly Fernandez is singing isn't there some digital asset that's being created there some intellectual property that's being developed of course there is and we want to be able to make sure that Melly Fernandez can benefit from that and that's a big part of the property well years ago there was coming out of Harvard law school there was a group called Second Life and they started building up this whole notion of let's see if we can take law and play with it in a virtual environment and a lot of people played there and law firms were built inside of Second Life and a lot of people ended up with skills and of course the gamers as you know are very successful in these online environments at very high rates and guess what intellectual property disputes started to arise the woman who ran one of the law firms inside of Second Life is now running our music projects she's brought all types of contracts enforcement mechanisms that she's built in the Second Life environment over with us and now we're partnering with a group called the Accord Project and with the Integra Ledger to take those what we call fair trade music models which you'll hear more about in a few seconds and put them into the blockchain in the form of smart contracts let me just so the piece stones process which I'll just tell you very briefly essentially has us going into those communities and identifying the local musicians and giving them the opportunity to take their music and put it up online the most important part of this particular process is the ability to give people there the opportunity to to get out of their own way the current problem that we're working with is in the Cox's Bazaar in Bangladesh where 700,000 Rohingya Muslims have been transferred from Myanmar into Bangladesh and we're giving technology tools so that they can use virtual currency to incentivize each of them to be able to govern the camp itself as they may not be in a position yet to develop some intellectual property though we're working with some to do that there so the fair trade music model these are the kids in the Bangladesh camp that we're singing allows them to benefit from the social justice fair trade music platform and then through innovation they will be able to gain their own identity how can you all get involved in this we decided hyperconnected means that each one of us needs to reach out to you the idea that you're raising public awareness in this environment of the problem there is not enough each one of you can be a digital pen pal to some individual in those settings and work together every one of us is going to need an identity and we can learn from each other so part of what internet bar is doing is establishing this kind of a program so that we can develop that there and as you can see radical collaboration gives us the opportunity to bring a lot of people together all at once to do this thanks thanks Jeffrey so we'll move on to Karen I think they just have to I'll get started so good afternoon everyone thanks to the organizers for bringing us together I work at Physicians for Human Rights and maybe just by way of background until the slides come up Physicians for Human Rights is an international NGO we use science and medicine to document human rights violations around the world and yep and we have used the evidence that we've collected in various investigations to help support accountability mechanisms in many many cases around the world we were established more than 30 years ago and over the course of doing this work we very quickly identified that sexual violence cases were some of the hardest cases to prosecute as many of you know it's very very difficult for survivors to even dare to come forward certainly in the context of a conflict or fragile environment or even post-conflict context the stigma is so great from their families, from the community and there's incredible risk of reprisal among many many other hardships but what we identified is that there are still cases of survivors who are willing to take those risks and have their day in court and this photo that you're seeing 14 survivors who wanted their day in court in DRC to face the accused what is so devastating about so many of these cases is that they fail because there's a lack of adequate evidence to support the allegations of the survivors so many of these cases fail because there are numerous obstacles to adequately gathering evidence to support these prosecutions forensic medical exams are rarely conducted in many of these environments people are not adequately trained medical charts fail to document meaningful findings that could help inform investigations and prosecutions there are very few if any storage facilities at a medical facility or at a police station or even in court there's a lack of clarification about the legal law enforcement and legal communities about what each of these professionals roles are within the course of a justice process and there's an absence of a shared language as you all know even from our work in this country and your work in other countries as well the medical community has their own professional jargon the law enforcement community and legal community have their own professional jargon these communities can truly understand their counterparts from other sectors so all of this goes to the notion that there is a need a serious gap in coordination across all these sectors to work on sexual violence cases so in 2011 we established the program on sexual violence and conflict zones at Physicians for Human Rights the goal of the program is to help the process for the police and clinicians, doctors and nurses, lawyers, judges to be able to more effectively collect and document and preserve forensic evidence of sexual violence to help support local prosecutions of these crimes and also the critical component of this work is to be able to cultivate networks of collaboration among these professionals so that they could work together with each other more effectively on a case by case basis to better support the survivor at the center of this process we do this work in Kenya and in DRC and we started to also pilot this program in Central African Republic I'm going to share with you some updates about our interventions on the low tech side and on the high tech side and how this is played out in court so we run very robust training workshops on low tech capacities that clinicians and police and lawyers and judges can do within their communities with the resources that they have so how to ask better questions during a patient encounter how to do a full head to toe exam for example in order to do a proper forensic medical evaluation we also train on how to do pediatric evaluations pediatric cases are very common in these communities we train on crime scene investigations how to document a crime scene, how to collect evidence and package it and maintain chain of custody we also focus a lot on documentation documentation is critical to do our work in the Democratic Republic of the Congo we saw that there was no standardized medical intake form to document sexual violence cases so we work together very closely with our colleagues from the medical law enforcement and legal communities to create a standard form to document sexual violence cases a forensic form and we borrowed from Kenya and from South Africa the United States, Canada, many other places that had a forensic medical form and our Congolese partners identified those aspects of the form that would be most useful to their context and they developed this medical certificate this medical certificate is now being used in the eastern part of Congo in military and civilian courts and it has had a transformative effect on cases going to court cases are succeeding more and more because there's more data that the clinicians are prompted to ask by virtue of having this form so it's a low tech solution but it is transforming these cases and the judges are the most thrilled to have this form because it helps them adjudicate these cases by virtue of having more data to help them in their decision making nevertheless we are functioning in environments where there are external factors that still make it very difficult to pursue these cases effectively so if a health facility is several kilometers outside in a remote area of the country it can be very difficult for the police to actually navigate roads like this in DRC and elsewhere where this is a common site because they don't have for example access to vehicles or even money for fuel to get to the health facility to be able to pick up this medical certificate so there are a range of infrastructure challenges that we have encountered that also present as an obstacle to justice so we started to identify that in places of low income countries, middle income countries certainly where we work in eastern Central Africa there's incredible mobile penetration in these communities and so we've developed MetaCapt it's a mobile app that pairs a standard medical intake form with photo capture capacity to allow the clinicians to actually document their findings on a digital platform and to be able, this is what the a screenshot of what it looks like so they're able to document their findings and identify which parts of the body the injuries are sustained and it prompts clinicians to take forensic photographs during the patient encounter the app can work without Wi-Fi connectivity the clinicians could use this and create as many records as needed eventually to upload it to the cloud they do need Wi-Fi connectivity to be able to upload it to the cloud but one of the big innovations that we have that we have developed through working on this tool is how to also in a paper full environment how to be able to provide a copy of the form to the survivor at the center of this process and so we had to develop a mechanism to allow the clinicians to be able to print directly from their tablet or phone to a printer without any electricity without rooting the data the confidential data through the internet to be able to allow the survivor to take a copy of her own certificate home with her and then the police and justice sector officials would be able to access the data from the cloud and it's been very challenging to determine who should have access to what data because the workflows are very different in very different communities but the idea is that the data that's gathered in the medical facility can directly inform the police investigations and the prosecution we have developed the tool, the app using collaborative design or co-design that's the ethos of working with the end users as we design and develop this tool so we are constantly working with our partners in DRC and Kenya feel testing the app getting their feedback what's working, what's not working for you what do you need to be able to be valuable for you at the end of the day when this product is completed and so this has been an incredibly helpful tool for us to learn about their needs but even more effectively for them to feel like they're invested in this process too and this tool they can directly see their feedback and input into the final product we're hoping to build out this tool to be able to allow us to do aggregation of de-identified data so that we can do crime mapping to be able to use the data to identify for example perpetrator troop movement or what uniforms the perpetrators may have worn languages spoken or the injuries sustained by the survivors all of these items are really critical for tracking trends or patterns in the violence that are critical helping to support allegations of crimes against humanity for example where we need to identify evidence that can support widespread or systematic crimes taking place it's also really critical to show that who may be in charge as command responsibility for these cases I just want to end by identifying how this has played itself out in court in Kovumu a small village in DRC we started to see cases of young girls some as young as 18 months old to 11 years old being targeted in really brutal cases of sexual violence they were taken from their homes in the middle of their night and brought to a field and brutally violated and abandoned there eventually they were taken to a nearby hospital a pansy hospital where they were treated and the clinicians who we had been working with documented every single one of these cases using this medical certificate over the course of three years 42 cases of these young girls came to the hospital they didn't happen all at once but they happened systematically and all of the girls had the same mode of experience of the violation and brutal brutal injuries it became very difficult for this case to be investigated because it surfaced that a very powerful militia group was responsible for these crimes and the leader of this militia group is a sitting member of provincial parliament in South Kivu so it took years but with advocacy with the community with the medical and legal professionals we worked to have his immunity stripped and finally it was and he along with 17 other members of the militia group stood before court last November evidence was collected and beautifully labeled and victims and witnesses went very strategic innovative protections but ultimately in December of 2017 11 men including this lawmaker were convicted of crimes against humanity for rape and murder ultimately we see that small steps can actually have a big impact and so it's been very heartening to see how medical and legal community by virtue of working together can help to secure greater likelihood of success and prosecutions but really significantly help to bring meaningful access to justice respect and security to survivors and the communities affected by these crimes thanks that was quite compelling so next we have Antoine good afternoon everyone thanks to the organizers for inviting me to share our experience the sometimes I get asked why what's the name Ulula refers to the relation that women make in East Africa and Northern Africa and other cultures for funerals of weddings because the idea behind Ulula when I started it five years ago was how can we give a voice to people that are typically invisible or completely silenced in supply chains or in the deep in emerging markets that came from working ten years in these countries and thinking that I was the white men preaching things that were never going to happen and I saw technology as a means to an end not as an end in itself I'm an economist so I have my own problems but I didn't drink the Kool-Aid of technology if you will so I started Ulula about the time that the Rana Plaza happened which is something that is typically vivid in your memory where a thousand and a hundred Bangladeshi workers died in the collapse of the Rana Plaza complex and the idea that we had I mean was broader than that actually we started with ideas around raw materials which I knew best and I still have we have quite a bit of experience in with the idea that we would use very simple mobile phones so very similar to you that you needed to go super low tech first to provide a way for workers and communities to have a voice in the way they were treated at work or the impacts that other big companies may have on them so we created this mechanism that can do that in a way that I'm going to describe the background to that is I think Ambassador Dibacca gave a very extensive overview of the problems that he has spearheaded but the backdrop is 40 million people are victims of modern slavery a lot of them are women and so I have real problems in getting their voices heard and that's a big weakness if you ask a company whether there is modern slavery in their supply chain if they say no it's probably that they haven't been looking that doesn't mean they are really bad that means it's a very complex issue but it's a pervasive problem that touches many goods and happens in the US, happens in Bangladesh, happens everywhere so it's not something that one can feel safe about it's really a pervasive problem that we're trying to tackle and I think from the perspective of this symposium on the one hand I think we've seen a lot of progress in terms of legal regimes especially I would say in OECD countries with the modern slavery act in the UK now there is a new regime coming up in Australia there is a new law in France so you see this burgeoning of legal reforms that are very helpful in many ways because they are I think making the cost of modern slavery higher and changing the incentives but on the other hand I think enforcement is broken and I think that's where technology is as a role to play self enforcement through audits is something that is very limited for a range of reasons I mean the ways in which audits are conducted is flowed with a bunch of workers that are supposedly anonymous but not quite so for a range of reasons it's at best a picture in time and it doesn't give you a sense of the changes of consensus and the way these are conducted is always, I mean typically announced so that's not conducive to a very good sense of what is actually going on in a factory in a plant etc and so that's the backdrop of Fulula and on the other hand you have governments that are in an ideal world would play a role but typically you're talking about places similar to what you were describing where the rule of flow is poor and difficult to enforce so what is Fulula in all that I mean the solution as three core modules and that are here it's really the low-tech and that I'm presenting we can run surveys in over 130 countries using the best voice social media and the web so we're trying to be omnichannel we don't try to say there's one that is superior to the other the one thing that we're not is an app the reasons for that is that you have still 50% of the world that is offline and you have app retention apart from friends on this panel having a big problem in keeping people on their apps so we felt that staying simple staying native was the way to go and really trying to develop two-way conversations so surveys is obviously one way that's very simple but having grievance management has been something that we started working on so that we can raise cases understand when there is gender violence or harassment or other types of abuses of human rights or labor rights and and just broadcasting in terms of awareness is the third leg of that tool that we have now all of that is super low tech but we can do that at scale in other hundred countries and that's powered as well with a partnership that we have with IBM to use what's on in terms of data analytics and now we're getting into the blockchain world which I can talk about but I think it's baby steps at the moment where we are seeing how can we provide more credibility to the human side of what's provided on the blockchain when you're encoding the provenance of the goods. So I'll finish with three examples quickly one is we've been working with quite a few corporations and here it's just an example in China where there is definitely government capacity to enforce but actually 80% of the audits in China are false for one reason or the other there is corruption there's a number of reasons why the information that you get is not quite right so that working with these corporations we've been working to crowdsource information directly from workers and to understand what were the problems what we've seen in terms of we've done that through voice and so that limits if dropping we were not using WeChat in that sense because there are also some problems with WeChat in China we can use it in other settings but this one was required I think voice and we found out issues with wages with health and safety that didn't pop up in any audit report before that led to some actions in terms of health and safety training in terms of changing in management practices and some considerations on wages that are I think much more complex to tackle another example that expands the scope of what we have but looking at human rights more broadly is we're working with extractive companies in various countries to basically get the pulse of their social-environmental impact so that they don't get in trouble because they typically do because they really don't understand what their impact is and so we've been working in South Africa Chile, Peru and Brazil to basically get a continuous pulse surveys of what is the impact of a mind on health, education, safety dust, etc. a range of indicators using SMS and voice again and feeding that data back and flagging when there are issues to the company with fairly strong participation rates because we use small airtime incentives that do work in this context and you're talking about rural, Peru things that are super last mile deployments. A third example before I stop is taking a different perspective. What we do is not really working when you have multiple parties coming up. I'm going very fast on these examples but very often we are just a technology we are not responding to cases where because nobody should trust me per se. I don't have 10 years of experience working in Peru or in China, etc. You need to have an example that a worker can trust. They speak the language they know that these guys have stuck their neck out for me and I'm just there to provide a mechanism for them to do their work more effectively and at scale. And in that case I think the impact that came from the government so with collaboration from companies and other NGOs to tackle problems around again issues with not just mining but mining telecommunications infrastructure that the government was unable to and still has struggles to enforce when it comes to the social impact consultation with communities and where our system is able to provide information to citizens, register the grievance and then connect with the companies about the problems that are facing when you have environmental impact assessments. So these are a few examples that I wanted to share and I'm happy to discuss more. Thanks. Thanks. I don't have any slides so it'll just be me. Thanks for inviting me Hamid. I work on strategic response at Facebook. We are a small team within product policy. Our product policy team is the team that's responsible for writing our content policies of what we allow at Facebook and I work on a small team that works cross functionally a bunch of across a bunch of different organizations at the company including legal and operations, cybersecurity our counter-terrorism team, etc to deal with any number of content related crises that we have at the company. So I briefly want to talk about a few things. First our community standards then misinformation and false news our work around elections and other important changes that we've made at Facebook recently. So our community standards we recently published our detailed community standards that our reviewers use to make decisions about what isn't allowed on our platform and we put these detailed standards out for three reasons. First we wanted to be absolutely clear that harmful language images and videos have no place on Facebook. And then second we believe that increased diversity on where we draw lines on complex and evolving issues. And then finally we hope that by sharing the details this will prompt more open discussion and honest dialogue about our decision-making process. Our community standards are global that means what applies here applies in every other country around the world. Some people in Brazil don't like that because it means they can't post things that are in other places and people don't necessarily like that because there are things that people post that we allow that they wouldn't necessarily allow. Our content reviewers undergo extensive training after they join and also after they've been at the company for a few months they are regularly trained and are trained when our policies are clarified and also when they evolve because policies change all the time. Our duty policy for example has changed throughout our history. We make mistakes all the time and when we are made aware of those mistakes then we change our decision and we also make sure to retrain the person who made the mistake to make sure that it doesn't happen again. So next I'll talk about misinformation and false news. I think it's important to note that we at Facebook we don't arbitrate the truth. We've talked to a lot of experts and they agree with us that they don't think that Facebook should be in the business of deciding what's right and what's wrong. That said over the last couple of years we've been working really hard to reduce the spread of false news on Facebook through a combination of machine technology and also human review. With misinformation we have to fight bad actors, bad behavior and also bad content. So by bad actors I mean fake accounts, foreign agents, spammers, scammers, hackers and repeat offenders. By bad behavior I'm talking about polarizing, misleading, spamming, impersonation, sensationalizing or engagement bait. And by bad content I'm talking about false news, hate speech, spam, graphic debate and links to low quality web experiences which we refer to as ad farms. To better communicate our approach to misinformation we actually commissioned a short film it's about 12 minutes on our website called Facing Facts and it's an inside look at Facebook's fight against misinformation and how we are marshaling the forces against it. Misinformation is nuanced and really a complicated and complex and we're combating it from multiple different angles. We released recently an updated news literacy campaign that provides people with tips on how to spot false news and also how to report it. We're also measuring our progress in our fight against misinformation but we feel that it's critically important to do this. So in April we announced a new initiative to help provide independent research about the role of social media and also its effect on democracy more generally. I'll talk a little bit about Facebook's strategy for stopping false news. So at Facebook we identify false news as news articles that purport to be factual but which contain intentional misstatements of fact with the intention to arouse passions attractive viewership or deceive. False news has been a tool for economic or political gains and we're seeing new ways of it taking shape online. Spammers can use it to try to drive clickbait or to yield profits and it's been used by adversaries in recent elections as well. False news is bad for people and it's bad for Facebook but we're making significant investments to stop it from spreading and to promote high quality journalism and news literacy. So we've got misinformation on Facebook has three parts remove, reduce and inform. So we remove accounts and content that violate our community standards or our ad policies. We reduce the distribution of false news and authentic content like inauthentic content like clickbait and then we inform people by giving them more context around posts that they see. This approach roots out bad actors that frequently spread false news dramatically decreases the reach of the false news and it helps people to stay informed without stifling public discourse. Another tool that we have to reduce the spread of false news is that we've been engaging third party fact checkers all around the world to let people know when they are sharing news stories excluding satire and opinion that have been disputed or debunked to limit the distribution of those stories that have been flagged as misleading sensational or spammy. We demote articles found to be false by fact checkers and news feed causing them to lose about 80% of their traffic. We now work with independent fact checkers in the United States, France Germany, Ireland the Netherlands, Italy Mexico, Columbia India, Indonesia and the Philippines and we're also willing to roll out new fact checkers around the world. Next, I'm going to talk about elections and our goals for elections at Facebook. We want to help people. We're tackling a wide range of election integrity issues on our platform to ensure that people have accurate information about elections. They feel safe expressing their opinions and that they are motivated to participate in the electoral process. We also want to make elections better by preventing foreign interference which remains our top priority. We must also tackle a wider range of election integrity problems proactively like false amplification. We also want to improve election discourse on the platform. We take action on fake accounts, disrupt economic incentives and prioritize information posts and down rank unauthentic content including false news and misinformation in news feed. We also want to build useful tools so our civic engagement product priorities focus on various times people can use Facebook or might use Facebook in an election cycle for example telling somebody when they can register to vote or reminding somebody that election day is coming up. We want to find ways to do more as well so we're investing heavily in people, boosting technology proactively looking for harmful content bringing more transparency to all of our ads including political ads and working with the broader community. And then we're tracking over 40 elections around the world. I'm part of one of the teams that does that. In recent months we started to deploy new tools and teams to proactively identify threats to the run up to specific elections the latest was the referendum in Ireland the abortion referendum in Ireland we first tested an effort during the Alabama Senate election and planned to continue these efforts for elections around the globe including the US midterm elections. Last year we used public service announcements to help inform people about false news in 21 separate countries including in advance of the elections in Kenya France and Germany. So really quickly I just want to end with one of the challenges that we've made at Facebook recently we've made these changes to prevent bad actors from using misinformation to undermine the democratic process. This is a problem that will never be solved we're going up against determined well funded adversaries but we are making steady progress and here are some of the important changes this is only a small sampling of the things that we've been doing recently. Better technology over the last year we've gotten increasingly better at finding and disabling fake accounts. We now block millions of fake accounts each day as they try to create them and before they've done any harm. We're also significantly investing in security. We're doubling the number of people working on safety and security from 10,000 last year to 20,000 this year. We expect these investments obviously to impact our profitability but it's important to do at Facebook your safety or people's safety needs to come before profit. We're also collaborating within the industry. We recently joined 34 tech and security companies in signing a tech accord pact to help improve security for everyone around the world. As our CEO and founder Mark Zuckerberg recently said in front of in Congress last month at his testimony we take a broader view of our responsibility as a company. That means not just building products that will help people connect but also ensuring that they are used for good and not abused. We still have a long way to go and we will keep people appraised of our efforts as we go. Thank you. Thanks, Aide. So we're running behind schedule. I just wanted to take a few minutes to sort of engage the panel and some questions and then we'll open it up to the audience who I know have lots of questions. Obviously we have a very diverse panel here. People working on very, very different kinds of projects related to rule of law and technology. So I'm going to throw out the questions that I had prepared based on what I heard maybe for the first two speakers. So you presented some really, I thought, compelling applications of technology and how it could help. So is technology the solution? If we had unlimited amount of technology would that solve the problem? In your case you're having people who are recording instances of horrible crimes and so on. So is technology the solution? It can help in reporting cases of injustice, but could it also serve as a deterrent? So would love to hear your thoughts on that. Yeah, so I think that what's really important about the work that we're doing with MetaCapt is that we're not developing this app in a vacuum. We're very much focused on the low tech interventions that we've been doing in these communities to work with colleagues to understand what is sexual violence in the first place. And it's amazing what kind of attitudes and what certain beliefs are in terms of what actually constitutes these crimes. So there's so much work to do on the low tech end just even in terms of getting people's skills to a place where they can actually technically do forensic exams and to be able to compare it with the police and who's that trusted person in the police station. There's so much work to do on that basic level that I would say we have been very lucky to roll out MetaCapt within an environment where we already have these trusted relationships with these professionals. I would not say technology is a panacea. I would say it's one more tool in the toolbox. There's so many concerns about security and risks even for on the low tech side for these people the clinicians, the police officers who are documenting these cases they themselves receive a lot of risk and threats to themselves, to their families for doing this work. They're documenting mass atrocities sometimes that are perpetrated by the governments or by militia groups in these communities. So I would say technology offers another angle to gathering this data. It may help to bypass some of the challenges infrastructure hurdles in these communities. It's definitely not a panacea and it's one more tool that these colleagues can use within their toolbox. I would agree 100 percent. Our work is always starting from the ground up and starting from the ground up means you're building trust with groups as you go in. In most cases we will go into a community and there will be nurses or doctors there before we are. They've already established trust in that community and the idea that there is hope for somebody to be financially included in some form of economic opportunity is first conveyed to them who convey it to the people on the ground so that we get a chance to get started. But really what you're doing is asking people on the ground to define what does what do they want for justice what is it in their heads that they mean and you give them some tools and then they begin to play with the tools and show you what they need to get done. So that's how we start. The bootstrapping effect. Antoine maybe a question for you so the supply chain having people in China report things abuses whatever so is is there a concern about people who are reporting this type of data retaliation against them and the flip side incentivization also how do you get people to report regularly if you really want to analyze and get some outcomes out of this but any thoughts you have on those? We always undertake a risk appraisal before we start a project and there are projects where we have turned down and said it's not even possible to get into this because the environment was just not going to get anywhere we would put people at risk and even if you're not thinking about jurisdictions where things are tough typically if you don't have the institutional infrastructure to do something with the data then the question I ask is why do you want to collect the data in the first place be it a grievance or a survey if it's that there's really no business for me because six months from now nobody's going to participate and they're going to shoot the messenger the value of technology especially in supply chains and in a number of our businesses is that you do that over fairly long periods of time that's where you can get at scale of the workers etc it's not magic one that you just deploy on the ground and then suddenly miracles happen so and we are very mindful of the kind of data that we collect we try to collect as little personal information as possible we have a number of things to try to avoid any revelation of who has said what anonymity is something that is fairly we have a number of processes and technologies to prevent that but turning the flip side of that on the incentive side there are small things like airtime incentives that we can provide to that boost participation in the first place but eventually it comes back to what sort of co-design do you put in place to make sure that you're actually addressing a need and not just addressing a fad and then is what is the capacity of these different organizations and what level of support you can have so it's not always easy I've worked in India with NGOs on the textile sector and I would say they are fairly advanced compared to some of the NGOs now we're starting to work in the DRC where the mandate is there the wheel is there but the capacity is lagging and it's an endeavor for sure one question for Zaid questions so basically Facebook a lot of the business model is really based on data mining people's posts and what their interests are and therefore you can target things more effectively if you know a person's particular interests likes that's a lot of the business model of Facebook is there any people who want to use Facebook but who still want to have their privacy and not have their data mined like that to maybe opt out or a paid service what are Facebook's positions on that so I wouldn't say that our business model is data mining I would say our business model is selling ads and I think the way that we make sure we target the ads for the people who are the people who purchase our ads want them to be targeted to is by looking at what people or how people use the service and what kind of information they put on the service or on our platforms rather I think that so this question of whether of whether Facebook was looking at a paid service my understanding is that was a question that Cheryl Sandberg was asked in the lead up to Mark testifying before Congress and I think what she said was that's not something we're looking at we would have to consider it but she didn't mean to imply that that's something that we were doing and as far as I know and you know I work at a company now with 30,000 people so what do I know as far as I know paid like a paid service is not something that Facebook is thinking about so we have some time and we're going to throw it open to the audience if you have any questions we have two mics on either side of the room so anyone who has a question please why don't you come up to the mic yeah and please introduce yourself yeah my name is Pablo I work for the U.S. Department of Labor and I have a specific question for Antoine when you talk about this you know new platforms are using technology with work organizations like unions or private companies how do you present the idea of them benefiting from this technology because you know most of the time you can get defensive they can get defensive on you're talking about these issues that are running into the supply chain so how do you appreciate the idea of using technology to improve their services to workers or their shareholders do you want me to answer yeah go ahead so I've the pitch is different to different audiences you mentioned unions I think historically unions have been suspicious of what we do because they see that as potentially undermining their their intermediary role my answer is always that at the moment I would say sadly you're dying slowly and because you are you're losing membership and so this is just a tool that potentially helps you connect, reconnect and increase your base and I see that there is a younger generation of union leaders getting more and more interested in that and we've we've had some of that so I do feel that there is there is a message to these working organizations that is quite powerful in terms of being able to reconnect to your base have greater accountability and having better reach and communication with your workers when it comes to the companies I think it's it's a tool we have two segments some that are highly susceptible to operational risk so we work with extractive companies whose reputation is not necessarily very positive already so what they care about is operational risk and human rights risk so what is going to shut down their operations and that's the business case for them and then you have consumer good companies for whom it's not so much the penalties that they may get from the French law on non-financial reporting it's obviously the reputational cost and the cost of being dragged into courts that really drives them for the bigger ones I think there is a small cohort of companies whose DNA is really into that have decided to differentiate themselves by being super responsible like Patagonia and many others now coming up in the textile industry so for them it's a differentiation it's something that comes from a different angle but the risk is what's driving that in the first place and we're trying to make efficiency argument to them as well because the cost of audits is I mean it's expensive and it's also a waste of money most of the time if you're getting results that are not credible but that's also a very contentious question I mean if you had an auditor here I probably would have been more diplomatic about making that statement as well any other questions please yeah please step up to the mic yeah or the mic will come to you thank you I'm Ann Lo from the State Department I just had a question for I think it would be best for Karen and Antoine but interested in all of your feedback on it I was curious on your projects after I'm assuming they're funded funding that when you go into the country are they sustainable after the donor funding stops so for instance on the app the medical app to the country take ownership of it fund the people and what's your success rate and how do you make that transition from this is something that's incredibly cool that's incredibly useful to you that we'll give you we're funding to fund this on your own and how do you make that transition is it a smooth transition or is it abrupt and is it successful or not what are the best practices thank you we are always trying actually to convene gatherings with funders to explain to them that these are expensive initiatives and they require commitment to sustainability so that's in terms of the start up and launching of these initiatives and actually one of our biggest donors that has helped us develop metacapt and certainly I should say even on the low tech side has helped us carry out a lot of this work not exclusively but has been a big support as a state department but the app is designed to not be owned by physicians for human rights the data collected is not owned by physicians for human rights the hospitals and the countries where this data is being collected and of course the data is gathered with the informed consent of the survivor so there is a very rigorous process for obtaining informed consent for the survivor especially for sure even in the context of doing a forensic medical exam but with the added component of the technology and what that even de-identified data will go toward whether it's for research or advocacy or even to the police and justice sector so I think you raised a phenomenally important question at this stage in terms of our engagement with these countries is we work very closely with the ministry of health in Kenya with the county officials where the app is being rolled out and with the hospital administration the same is true in DRC where we've been rolling it out because the idea is that we're trying to field test this not only with the end users but with the people who care about statistics the people who care about monitoring and tracking these cases the government officials who are directly engaged in this process because it will be their tool as it sort of gets launched and they own it at this stage we're still running the initiative because we're just field testing it but it's absolutely the goal is that they will be running it and what happens at that point and the risks associated with that are issues that we have to work out with them in terms of protocols and troubleshooting and security I have a quick follow up to that also what about the private sector and this is for any of you can you engage the private sector also to scale up these efforts for us I mean from the beginning we started the company as a company not as an NGO because for instance even in the audit industry for the textile sector is worth one billion dollars and as I argued I think you could think that shifting these resources away would not necessarily you know result in a massive loss of welfare for the workers so you have I think you have money that is being wasted companies that are still subject to reporting and so they are checking the box and spending that money so we're helping them do that and we think there's actually a very strong case there when we work with extractive companies one day of closure in the mine is a million dollar so if you can you know have a somehow an argument that will help you reduce that by a fraction over the next 10 years you have a strong case as well so it's there's a strong in our case with the supply chain in the private sector there is an elephant in the room in terms of virtual currency with the number of internet coin offerings that have been out there and each of the examples that were just given there are a group of stakeholders the people created the technology and so forth that if you put a ring fence around all the people who are involved you could be using non-fiat or private currency virtual currency to be allocating it to each of the groups as they do the work of course this is new it's being tested and it's not the bitcoin approach there are going to be a lot of problems with internet coins because there are so many of them but the ones that are doing the business models right now are looking how do we create an economic relationship amongst a small group of people so just take the Bangladesh camp for it to self-organize you need the mayor of the camp you need people who are going to go out and do training in the camp you're going to need people who are going to take and do outreach all kinds of roles how do you create an economy without there's no money so you have to give tokens this is not a new idea Edgar Kahn years ago for domestic violence victims in this particular jurisdiction put domestic victims to work after they did the work and then he went out and found funding for it separately and elsewhere so the idea that you can incentivize people to work without literally handing them cash but then finding ways to support it later is going to be developed in this course this year with the virtual currency any other questions from the audience hello I'm Abraham Calone my question goes to Jeffery I like your your comment all your comments about innovation development and how you talk about internet and how people with creativity can benefit from internet I would like to know what's your interest in expanding your scope of operation towards West Africa if you have interest like Blangadesh or a certain part of the world or are you intending to extend into West Africa like Liberia because there are a lot of innovations in Liberia or West Africa that is growing rapidly but there are a lot of challenges when it comes to e-commerce a lot of people who got skills and talents musicians producers directors they don't have the means to really tap into internet expansion to grow globally benefiting from internet so I would like to know what are you also thinking about as standing your scope there as well it feels like a setup question we were in Monrovia in February with the team there and we sent a group over of musicians who then engaged with the State Department a lot of young people on the ground who were not only establishing what roles they could play around a musician because around a musician you have graphic artists, you've got web developers you've got accountants, you've got managers you have a whole team of people and why can't you take those skills and then make them available to a company somewhere else in the world that could benefit from them and be able to earn a living so we were there we engaged about 60 people on the ground there and we'll be going back as well and the same idea that we'll be doing in Hey, See, Favorite Zoo where we were a few years ago and then if you do this radical collaboration if you take these basic concepts and you work with a group like the World Justice Project which is in several countries around the world all at once with hundreds of projects you just get the ideas out into best practices and make them available to people on an open source basis to use that's a little ways down the road because you have to test them and work them out first but certainly Liberia is a place that's on our list to go back to again this year Any other questions from the audience? Maybe I have one more question for Zade so you mentioned that you know, we all know that Facebook operates all over the world and you mention a whole bunch of countries also where you have fact checkers and all of that given that the way that Facebook is used is so different in different countries like in some countries I've heard that it is the internet that's what people use is there any adaptation that you would make in situations like that given that the nature of the use is so different in different countries? So there's not an adaptation in the way in which we would apply our policies but what we are looking to do in making sure that we're doing is that we're allocating the right amount of resources in different places depending on what kind of issues they may have there so so we, so you know Myanmar as an example we now have a number of teams across the company across the world that are now more deeply engaged on issues in Myanmar than we had before we're also hiring a lot more people who are native speakers to work on issues there so depending on what our needs are then we would recalibrate and to see what we need to do to make sure that we're addressing issues in each country but our policies apply worldwide So we have time for a one minute and strictly one minute closing statement from all the panelists you know think of what is it that you want the audience to sort of walk away with and remember after this session so Jeffrey go first We're in a hyper connected world the rule of law operates out of the 19th century we need to start creating new ways to make rules from the ground up we need to trust in them for international standards while that's happening why don't we connect using social media applications such as the tools that so many people in the developing world use I believe it's a Facebook application where translation takes place and you connect people one with another so that I'm connected with one of the artists in Bangladesh from Myanmar that I'm working with and she knows who I am and you do that around the world simultaneously that's what I think is a step in the right direction I would say that technology has an incredible power to support people in the work that they're doing and also it can be very disrupting in very negative ways so where possible to engage respectfully with dignity with openness and transparency with these communities so that people in these communities have a role to play in being the protagonist in their own story as they choose to or choose not to develop technology that may work in their community I'd say that the purpose of our technology is for people to be able to exercise their rights so it would be ideological if the application of that technology was in any way or form denying this potential and the way to realize that potential is really to take a fairly humble and much more human approach to designing these tools checking how they fail because if the giants are failing small organizations like us will fail so I think we are approaching that in deep respect for the fact that the human infrastructure is what really matters at the end and what will give power to our small contributions Great I would say that technology changes very quickly I've only been at Facebook for nine months but the job that I applied for the job that I interviewed for the job that I started on my first day and the job that I do now and stuff happens things change our priorities change our resources we put resources where they are needed and it's interesting to see how quickly technology can change sometimes overnight and then the last thing I would say is that and I think that from what people have said I think most people on the panel would agree is that technology isn't good or bad technology is sort of agnostic but the way people decide to use the technology is what's good or bad are you going to put technology to good use or are you going to use it for nefarious reasons and we would hope obviously that everyone would put it to good use Great well the only thing I could add is I'm a technologist myself and I've seen a lot of very complex AI technology going into recommending better movies to watch and stuff like that I just think that this shows that there's a whole new world of applications that this advanced technology could be applied to and it's great to see that happening here already so it's three o'clock on the dot thank you all for coming and did you have an announcement Yes, just to quick it out to the panel and as the last panel is going to get mic'd up as it were and hopefully successfully I remind everybody that if you haven't had an opportunity to please fill out your evaluations you can deposit it right outside this venue but it will give you an opportunity and again thank you for all for your attendance and your insightful comments There are three here There were four Good afternoon everyone and thank you for your stamina and your interest in joining us for our last session this afternoon and your final dose of substance on rule of law issues My name is Andrew Solomon I'm a senior rule of law advisor at USAID here in Washington DC On behalf of the panel I'd like to thank JustTrack and the University of South Carolina for inviting us to present today and also for selecting this topic for this the third annual rule of law innovation symposium rule of law 3.0 if you will Each of the previous symposia have contributed significantly to the development of the field of practice by exploring innovative approaches practices tools to address both long standing as well as emerging issues and challenges to rule of law promotion around the world inclusive of both justice and security sector assistance and today's symposium has thus far continued this tradition with developing a better awareness and understanding of the role and the use of technology in strengthening law enforcement and also facilitating greater public participation and engagement in governance processes now our final panel this afternoon endeavors to continue that tradition and we will do so by focusing on how to improve service delivery through e-governance and e-justice systems and tools now this topic is highly relevant to governance and rule of law initiatives that we see around the world including many that are being supported by USAID by the State Department and other actors including maybe by some of you here in this field and on this panel many of these initiatives both e-governance and e-justice integrate these systems and services as a means to strengthen the ability of public sector institutions to deliver key services more efficiently and more effectively in addition they can serve to make these same services more accessible and more responsive to all citizens of society and then these systems are also used to strengthen the transparency and the accountability in public administration which in turn can contribute to curbing corruption and at the same time strengthen public trust in institutions of governments including justice and security and I think we may all be familiar with authoritative sources that demonstrate in many of the countries in which we work it's the justice sector and the security sector institutions the courts, the judges the police and in some instances the legal profession that rank at the bottom of public trust and confidence so e-governance, e-justice offers the potential to address or begin addressing some of these issues but it has some challenges and some pitfalls so fortunately today we have with us three experts seasoned experts to help us really better understand innovative approaches to these types of systems and tools and how to apply them in different types of programming you have everyone's bios but I would still briefly like to briefly introduce the panel beginning with Dr. Julia Glidden general manager at the global government industry practice at IBM global business services we then have Nino Vardo Sanidza senior legislative and oversight manager in the good governance initiative in Georgia this is a USAID program implemented by Tetra Tech ARD and then we have Jeff Apperson vice president of the national center for state courts which I consider to be a leading think and do tank in our field our panelists will begin with 12 to 15 minutes of opening remarks we will then shift into a facilitated discussion here between the panelists and we will leave some time to open up the questions from the audience for your engagement so with that Julia thank you Andrew on behalf of the IBM corporation it's my great pleasure to be here today talking to you about such a timely and important topic I think I was struck by the last discussion technology is not the answer that might surprise you coming from me in the IBM corporation but I could not agree with the previous comments more technology is a facilitator it's an enabler and in the current disruptive pace of change we're going to reach a tipping point where it will be negligent on us all not to embrace it so with that framework what I really wanted to do is hope technology works ah does anybody know how to Andrew? these are one of the pitfalls of technology sometimes there okay what I wanted us to do this late stage in the day we've heard about so many tremendous initiatives and we've heard about challenges and obstacles to harnessing technology to improve social justice to increase safety citizen trust in the validity and the scientificity of government services but I kind of wanted us to wipe our minds a little clear from everything we've talked about just imagine a world let's all go back to being 5 we have no boundaries we don't know what can't happen we don't know about the money roads in DRC we just imagine a world we're accessing public safety services legal services justice services really any service is as easy as talking to a friend the service is designed for me not 2 million people in a state or an entity or region or government and I access it as if I'm talking to Andrew right now because we know just by the very basis of the topic today right we know it's almost a cliche that the world has changed and that technology has fundamentally changed us all but what I've found in 20 years of working in tech innovation and government and I ran my own tech boutique where I tremendous amount of work with the United Nations and the EU on driving the type of tech innovation pilots we heard so much about today we know that in forums like this we take a step out into the way we lead our real lives all digitized all interactive all voice recognized all mobile all data but then we come back to our day jobs and it's like well you know not here you don't really understand USAID you know the UN has a reducing administrative burden but yes Dr. Glidden to participate in this panel you really do have to sign it seven times in the right ink the EU really will delay a project for six months because the 27 partners haven't done the right time but I remember when I ran my own company I had to actually get the bank to invent a watermark because nobody had it anymore they weren't used and this was for a project about you know reducing the administrative burden and driving effective and efficient innovation and government I mean it's hilarious so technologies changed us all it's changed all of our expectations I remember standing in that bank with the teller looking at me like I was crazy and I was like well if it says Barclays on it put the data and maybe you could sign it and it'll be good enough technology has changed all of our expectations about what's possible except when we go back to our day jobs and I can tell you working in a company with almost 400,000 people across 173 countries in the world we are as riddled by the hindrances I'm talking about as anyone because by the very nature of bureaucracy and I'd like to remind us how it used to be a good word when Weber coined the phrase it was in advance because it enabled scale on village living when everything was communal before the iron cage it was in advance we're all now stuck within an iron cage of bureaucracy despite the fact that technology has shown us new pathways forward and has really fundamentally changed our expectations of what's possible and I would say to you governments and international organizations not for profits NGOs they're no exception we are all as I'm talking I've worked on countless innovation projects with the UN, countless innovation projects with the EU, countless GovTech projects with real pioneering agencies and government actors and stakeholders around the world but when I step back and even hearing the great work that is happening in the field today when I really think about it what we've effectively done so far is basically we've digitized the status quo so we used to have to shuffle amongst a proliferation of folders and manila files and access pieces of paper and data but really what we're doing now is we're juggling multiple mobile apps if we're lucky and pieces of paper and files we haven't really fundamentally gone beyond the paradigmatic thinking of the 20th century of the 20th century bureaucracies we might create a mobile app but we still don't yet have a holistic vision and I tell you from my EU work particularly in the smart city domain really we've been stuck in a land of pilots because when you hear what MediCap is doing you think but of course why is that not the norm why wouldn't I'm going to say that wrong somebody chatbots these technologies are increasingly ubiquitous they're increasingly cost effective the ability to deliver them are increasingly there but really we're still in the land of pilots if I challenge the technology and go back the confusion of the different these apps are like little pilots everywhere we really have no holistic vision we really are still very much stuck in our 20th century inherited ways of thinking about when what citizens really expect and especially the ponderance of the world with you that are younger than most of us they're really expecting us to reinvent the way things have been done to cleanse our minds to be five years old and imagine a world because what we know for sure is we would not imagine a world where you had to stamp papers 52 times where you had to transmit the same documents to the same people the same time where we weren't able in a remote village the technology that is ubiquitous and is available to improve lives to keep people safe to provide social justice we wouldn't invent that world and we wouldn't invent a world of why we can't because I promise you everyone in this room as I'm talking is thinking that would be great but you don't understand can't happen here I want to give you one aha moment I'd had about the limitations of our own mindset in terms of what's possible and it was I met in Kazakhstan I met a venture capitalist woman who told who knew the Airbnb founders I was like wow you're gonna be rich someday she told me they were on their last $50 pitching for VC monies which they didn't get but they brought their beds to stay in a friend's room they brought their air mattresses and they said hey why can't we do this to make it easier and cheaper and more effective for people to just stay when they don't have money or to stay where they want to stay and they invented Airbnb this is six, seven, eight years ago and I remember thinking oh you know they don't understand insurance, health and safety the hotels, regulations but you know what these were 22, 20 year old kids they didn't know why they couldn't they just had a yes we can attitude and you know what I would like to argue is that it's not just time to harness technology to reinvent the future there's a moral obligation on all of us that are privileged to work in the jobs that we work in that are motivated by social good I joined IBM two years ago and it's still very hard for me not to talk about myself as a public servant because that was my entire career and I know it's a vocation you know we do what we do because we care and we genuinely want to improve the rule of law to make people's lives safer to improve social justice and what I think is that the technologies that we've heard about today chat bots, Watson, artificial intelligence blockchain we're at an inflection point in time where if we have the will and the imagination it's not about money anymore to change the paradigmatic people in process views that we've inherited that we operate in every day that we really can take these tools and technologies and bring them together to place people at the center of everything we do to make accessing tools of social justice and of public safety as easy as talking to a friend now I say here we talked about you rule of law 3.0 I would say cognitive government I know it's a term I've been talking about with the United Nations for a while because I think we now talk about digital government and that was in advance on e-government but where we really need to be going because this is where the technology is increasingly cost effectively and easily taking us is cognitive government and what do I mean by cognitive government I mean a government that's able to take these vast streams of data and again we heard about just the power of data in the DRC to bring social justice to victims female victims of horrific crimes but this power to harness data is capable across every element in our lives and every element of the rule of law topic for us here today one small pilot I'd like to just share with you maybe two or three but very small vignettes is AI what we call cognitive does is it allows you to access unprecedented streams of data on a volume and a level at a rate and a pace never heard of before but not just to access that data and stick it into algorithms that we then just extract out what we already know but AI cognitive can process that data and actually understand really more importantly reason is fundamentally disruptive in terms of reinventing not just transforming reinventing paradigms to learn so one small application of this was in the Florida Department of Corrections we all know that the quickest way to create a lifelong criminal is to incarcerate them you basically send them to get their Oxford PhD in a life of crime and we all know that the real crime is when we can sign a juvenile to that fate so with the Florida juvenile department the Florida Department of Corrections did a pilot where they applied AI or cognitive to access all the criminal record files and to help the judges in sentencing and making sentencing recommendations and the statistics were really as a small pilot I don't want to overstate the case but the statistics were really very very powerful 28% reduction in the number of children sent to secure facilities that's 28% of those children were not sent to become lifelong professionals and a lifelong career of crime 22% reduction in children that were put on parole and a 15% reduction which is really fascinating and children that were arrested in the first place and put into the system in the first place if you just think out of this one small pilot alone what you can do in terms of shifting the paradigm from what our 20th century world is today we are reactive and we are siloed our criminal justice does not talk to our police does not talk to the social workers does not talk to the schools does not talk to the hospitals so much of the elements of our pathway to crime are known usually by the age of 7 the school knows the child is out a lot or comes in hungry or has suspicious bruises the hospital knows when that child has been in and out of the hospital the police know when the parent that has been incarcerated for child abuse has been let home but what we don't do is join up these data sources to create a holistic view of a citizen of a person at risk to create a picture that will allow the institutions of government to intervene before the rule of law is ever even at the point of contact with that child and what I would argue is that again everyone I can hear and we had Facebook on stage talking about the challenges and their very real challenges data protection and privacy big brother those are very real we are not estimating those but if we allow that immediate oh no not here we can't do it to stop us from responsibly looking at ways to harness the ability of cognitive and data to genuinely reinvent the way we do things so that we are predictive and proactive not reactive in siloed we are losing an opportunity to transform society for the better and what I would like to leave us the ability to do this is closer than we think and becoming closer than we think every day because it's almost a cliche for me to now say the pace of change has never been faster we are living in an era of exponential change it used to be very easy when I ran my own company and to give a keynote like this to sound like I had a crystal ball or I was a genius because all I did is look at what happened in the private sector or how I led my life and for this it's coming in five years it's now two years and I would argue it will soon be 18 months so if we take a look at the data protection and the privacy constraints never mind the interoperability constraints and the legacy system constraints that held us back I would like to leave us with thinking about I know we talked about blockchain today and a lot of associations deal with cryptocurrency but blockchain I would argue our chairman Ginny Rometti argues is transformational because the internet was 20 years ago and it will fundamentally transform the way we can share data securely in a way that places the citizen the individual as the owner, as the custodian as the empowered owner of their own or steward of their own data because there is the cryptocurrency of blockchain open permissionless arguably a Wild West but there's also I won't get too technical but secure permission to access to blockchain where I control my data or on behalf of my child I provide access to those bits of a medical record of a police record of an education record to the stewards that I trust and I believe in to create a picture that can be proactive these tools and these technologies exist we've heard about co-creation when you combine the power of what's coming online faster and at a more rapid pace AI blockchain internet of things to give us even more data than we have with co-creation right harnessing the single greatest asset any government institution has which is the servant the individuals the citizens that it serves the ability for us to truly transform the social justice system to shift from reactive and siloed to proactive and predictive has never been greater and I would argue that the onus is on all of us in this room with the privileged positions we have and the jobs we have to imagine a future where no child where no person is left behind because of the limitations of our 20th century ways of thinking and reacting but rather we have painted a new vision of social justice based on the 21st century possibilities that are in our hands today and rapidly coming on stream thank you thank you for those forward looking remarks and I want to come back to the yes we can attitude the land of pilots and then also this challenge of sharing of information but before we do that I think Nino you'll bring us down to a more programmatic level based on your work in Georgia and here goes my course sorry I think I'm good so thank you Andrew for the introduction and I'm really humbled to be here with this distinguished panel and I don't think I've ever been described as a seasoned expert so thank you for that so as Andrew said I'm going to take you to a local level now and talk about some of the things that we at Good Governance Initiative in Tbilisi, Georgia have done to advance the e-governance in the country so just before we do that quick facts about where Georgia stands in terms of internet penetration and access but also some of the e-government efforts that e-governance efforts that the government of Georgia has undertaken so with the population of 3.7 million internet penetration rate is just about half of the population and this trend has been strongly growing for the last decade but there still is a quite large capital rural gap in terms of access quite large age gap and a slight gender gap as well with more men using internet than women and if we look at some international indices and take 2016 UN e-government survey Georgia has ranked 61st and 76th in the e-participation indices out of the 193 UN member states that have been surveyed so what you see here is a government of Georgia portal that combines over 70 e-services that it offers to the citizens of Georgia and you know we can say that Georgia has been quite lucky again over the last you know 10 years to enjoy quite high political commitment to ICT from the government side and there are some institutional mechanisms that are in place we have a digital we have a data exchange agency that is specifically tasked with advancing e-governance in the country and also with cyber security and government also has adopted the e-Georgia strategy which has the vision that you know Georgia can become a more efficient and effective public sector offering integrated and high quality e-services but to date the government efforts have mostly focused on providing these e-services mostly related to passport and IDs and some of the life events like marriage and childbirth and less so but still there are efforts to also move on to e-participation and several government agencies more so on the central level than the local level but they have introduced e-platforms for citizen engagement but although you know governments notwithstanding the fact that Georgia online service availability has dramatically increased we still have a gap in terms of availability of these services but also the use of these services so let's just talk about some of the enabling factors and when it comes to Georgia open government partnership has been one of the biggest factors in Georgia's journey in e-governance Georgia joined this initiative in 2011 and last year we became a chair of the initiative and I'm quite proud to say that we're actually preparing a global OGP summit that is going to happen in Tbilisi, Georgia this July and at the same time the government is working on its fourth OGP action plan so Julia mentioned co-creation and co-creation when it comes to OGP means that governments are working together with civil society to design and implement some of the commitments that the governments take and this has been one of the biggest cases for Georgia's journey really both on the parliamentary side and also on the executive government side co-creation has been something that has yielded the most impressive results in terms of what Georgia has achieved and we can say that OGP really has served as a platform to cement political will especially when we talk about the e-governance and the two tools that I'm going to present are actually a demonstration of the two commitments that came out of the OGP so this is the budget monitor tool and I believe we have a video queuing up if we can turn it on please because I cannot do it from here thank you thank you thank you I hope that you could read the subtitles because I only know one person that can understand Georgian in this audience so yeah the budget monitor tool is something that the state audit office of Georgia has developed with our support and also World Bank support and this tool has been operational for one year and it basically is a one stop shop for all information about public finances and it also is a tool for citizen participation in the oversight of public expenditures and the primary users of the tools are members and staff of parliament, media citizens, society organizations academia so here are a couple of screenshots of the things that the video did not show so here you can see how the tool makes it possible to compare how much two spending institutions are spending in the government in the employee compensation so here we see the constitutional court of Georgia and the central election commission and how much they're spending in salaries of their employees and then we see how many employees they actually have on board and the state audit office also has used this opportunity that the tool gives us to raise awareness about the sustainable development goals and also what role does the state audit office play in meeting the commitments of the SDGs and we can see that there have been 34 audits initiated by the state audit office that have been related to the SDGs and this page also showcases some of the findings and recommendations that the agency has issued and last but not least to talk about citizen participation is this citizens page so called which gives the opportunity to Georgian citizens to report cases of corruption that they have witnessed and the state audit office has the opportunity to follow up on these cases so the next tool is the government petition platform which is the first government e-petition platform it's called ichange.gov.ge and here with this platform citizens of Georgia can actually feel the petitions and the government is obligated to respond in a certain number of time so concept for the platform was also developed based on the co-creation principle of the OGP and with active participation from the civil society and to date around 150 petitions have been received and the platform has had around 15,000 unique users and it has been operational for six months now this platform has a challenge that is related to how many signatures does a petition require to actually be considered by the government and the number of signatures is 10,000 wish to put it in context the UK similar e-petition system also requires 10,000 signatures with Georgia's population being as I said so it is a real challenge to date out of the 150 received petitions none of them have achieved the 10,000 mark which continues to be a challenge and civil society organizations that we're partners with and our project as well continues to work with the government to make the number of required signatures less so that the platform actually achieves what it is designed for and then I would like to wrap up with some of the challenges to a participation in Georgia but I don't think that these are unique to my country obviously access skills and awareness speak for themselves when we talk about trust and confidence in successful examples it's important to note that Georgia has a very hard history and coming from a Soviet legacy trust in institutions is not very high so it is important when we talk about the I change platform with the budget monitor to demonstrate successful examples of what e-governance can actually deliver so that the citizens are motivated to participate and legacy of institutions also is on the same line with the state audit office this is an institution that was known for its punitive actions for decades and decades during the Soviet times obviously but also 15 years since then before the rose revolution when the country was swamped in corruption and then I already talked about the gap at the central and local levels which remains a challenge so with this I would like to wrap it up over to you Jeff thanks Nino those are some very good examples practical that we can dig into perhaps we can talk about how successful they are just think about that and also maybe we'll talk later about a little bit of political will you mentioned that as well so Jeff I hope you don't mind me referring to you as my justice ringer but maybe you could explore those issues but within the larger context of transparency and accountability yes thank you Andrew and it's good to be with JustTrack and back with USIP I started my career down the street here in 1982 and I remember I was so cowed by working for the federal judiciary and I met one of the first people I met was Chief Justice Berger he was one of my first bosses and at the time him being chair of the judicial conference of the United States he was worried about the impact of technology on the services of the judiciary and the challenges that we would meet and I always remember him saying one thing he was concerned about the values of justice he was concerned about preserving those values in the context of the technical evolution and in my 31 career in the federal judiciary and with the having managed the Yugoslavia Tribunal and the transition there and also worked with the national center for state courts and the staff like Jim McMillan here who used to manage the court technology conferences he said he was worried about the touch of justice and that the touch of justice in his view was you know established between the judges and the people and he didn't want to lose that so during my work with the judiciary I was always most interested in utilizing technology to enable the effectiveness in the extension of core values since that time in 1982 I served as a manager and we managed and also served on the advisory committees that managed the transition of the United States court system from manual to technical and there are so many lessons learned being one of the responsible people in the governance area of the United States courts there's a few challenges that I'm concerned about in the context of our evolution technical evolution and that is one of my our core interest is transparency and enabling transparency I think to a large degree that is one of the benefits of information management systems that are automated that are supported on websites or accessible by iPhone I think transparency is evolving as a general concept there's one thing I'm concerned about as we develop these systems I'm concerned that our emphasis is gravitating away from what I call the formal approach to justice which is largely used to be the way we found the truth and finding the truth is a core value for me seeking the truth, finding the truth and how we get there is one of the most important principles of an adversarial process or even an inquisitorial process might be how can automation affect the ability to find the truth that's what the judges do that's what the juries do that's what we wanted to do however the formal justice system is fading the informal justice system the alternative justice system is replacing the formal largely dispute resolution processes are no longer in many legal systems tied to the trial system they're tied to the plea bargain or the mediation or the new online dispute resolution processes but after a case is filed the public doesn't see the proceeding if it's in the form of a plea so yeah we're transparent but where did the process go I think those are things we have to think about as we talk about the values of the united states system and how it might affect the transitions in other countries the other thing I worry about is accountability and IT obviously information management systems gives us a better opportunity to monitor the performance of those that work in the system it allows us to generate performance management reports that facilitate our ability to track the management style of a judge or a manager or whatever it might be but it is enabling us in the long run to work more effectively number three I worry about case management disposition times and I want to use IT to enhance the ability to manage the case more efficiently one of the greatest tragedies in managing justice systems in the world is the time to disposition average case disposition times in most countries that are developing are over five years I know I've been in some it's 12 years I know one you know it takes 40 years to get through the system how can IT enable those types of issues now as far as implementation you know Jim and I you know I wanted to take our experience from the national center and the federal judiciary and transfer the knowledge and the practice to other developing environments and the challenges are as you know huge the capacity issues the institutional capacity issues the human resource capacity issues are huge and how do we effectively implement modern designs in non-modern societies that is a challenge and my thoughts are these first of all we have there are so many opportunities with IT not just in national development but we have to think about the trend in regionalizing services so when I think of development I think of East Africa or West Africa or the Caribbean as a whole I think of not only how can we implement a system in one country because our resources are so limited I think in NCSE thinks how can we manage more collectively a regional response that has the effect of solving some of our cross border issues that's an issue now I just want to go through what I consider some of our real development challenges first of all other goals for me was always provide enhanced public access to the record and the proceedings improve public trust reduce budget pressure and we talk about results one of the results and one of the incentives that I had as a manager was as we were implementing IT information management I had the budget I controlled the budget and I controlled the implementation I knew how much money I was saving when I introduced e-filing e-service, record storage internet services fee applications 40% 40% so when we had a budget reduction 10% in 2004 saved 40% now I'm not going to go through the whys but you have to be able to enhance productivity in fact it really also comes down to what is the management capacity of the operating environment you have to have inherent management capacity in fact the skill set has become so much more technical and you have so many managers with only a clerical background and you have to have the technical skill sets to make the transitions at the management and technical level and you have to develop those services now the other question is should we outsource or should we build internally I was always adverse because I believed always in protecting the institution and developing the institution and not developing dependencies outside the institution because as judges and managers we had to worry about conflicts I've always proactively felt that the key for us was we owned what we built and when we talk about collaborating with countries at the project level and we were talking about sustainability my thought is you start building sustainability from day one that's what I think and it's in several strategic areas and the strategic plan development at the management level the judicial level the civil society level is all part of the process but I wanted to say so also we have to understand the governance organization of every environment we're working in what are the control points who controls the decision making process and three who's the leader who's going to get the job done and who's going to be there when we're finished to oversee the continuing implementation of the project one of the keys of being NCSE is being an institution we now have projects directly with Nigeria and Trinidad as an institution so we don't have a deadline we don't have a project plan we're there from the beginning till it's in and we're sure it's going to be sustainably maintained and we will be a continuing partner with whoever we work with but how many countries do we work in where the court presidents or the attorney generals or the ministers of justice are only there one or two years in Brazil president judges one year Colombia, two years they have their own agendas so but if you have capacity that's institutional at the governance level like we have in Nigeria with the national judicial council chair being there for ten years you can get something done also another failure point in implementation every system has to run parallel in other words we have to have a goal of authenticating the electronic record as the original once that goal is achieved you can transition from the manual that's the failure point countries aren't able to transition from the manual to the electronic because they aren't confident in authenticating the record as the original also funding funding, yeah donor funding is one thing non donor funding is the challenge the budget process in most developing countries is not based on A OECD principles I always, I did not realize this until I did the budget study in Trinidad and I realized that cash based appropriations methods were very difficult it was very difficult to strategically plan when you can't execute until you get the cash big issue and I don't think people even understand that only 38 countries in the world have accrual based approaches to budgeting and finance and really so when you think about that if you're cash based you don't have outside funding you can't, it's hard to execute to what advantage can we take advantage of cloud technologies implementing I think is really important in the Caribbean for example we are operating out of a cloud forget where it's at Miami somewhere Jim but the server configuration is not a concern of the country that's operating the new software so you're able to build regional distribution around the cloud the other thing about technology is the reason we saved 40% was we were able to redistribute the work from the court to the bar in other words if you look at the process we're doing it twice we're only doing it once now so there's advantages in process implementation and then we get into the big issues and I'll begin it's four o'clock I'll begin to finish here but the cultural issues are huge and we all know that and we know we can't plan something as complex as a case management information management process with a fixed period of time there's one thing I learned we've learned in managing complex projects you set goals if you set deadlines and you miss the deadlines then you're going to lose trust I look at it and I told the Chief Justice of Nigeria we met the first thing I said she was talking about how she was going to improve transparency and public access with the new system I wanted to set a deadline and I said patience, patience, patience give it time it's not a strategic five-year plan it's just strategic generational plan and that also includes many of the elements that we're talking about here and many of the presentations are things that can be incorporated with these basic digital components of managing this process that I'm talking about but any one of the things that I've been discussing can create results that could lead to failure and I do know I've been to 90 countries, I've been in the courts I've seen successes, I've seen the failures most of the failures that are sustainable failures that are failures are because they don't have technical capacity and they try to run parallel and they fail and the computers are sitting there and they're going back to paper so I just wanted to bring up you know transparency is reinforced accountability can be reinforced I just got back from two work groups from the UN both of those topics IT can be a resolution, an enabler to improving transparency and accountability but I think what's most important for the courts because the first sentence of our constitution is to establish justice to me technology is all about that how do we establish justice how do we connect justice to the people for the country that's the key and it can make it work better it can make it work not so good that's all I have and thank you thank you Jeff provocative and insightful what I heard Julia saying we all hear different things but in many ways technology is propelling us forward to reality we have to learn to adapt to it and to use it Nino you're showing how it can be used in a very practical way for governments Jeff I heard you say well we still have some fundamental questions that we should keep in mind as this process unfolds and as these platforms systems and services are unfolding around this or being developed around this so we have about 25 minutes I definitely want to get the audience engaged but first we just want to maybe have a little bit of a conversation up here coming back to some of the issues I'd like to just tease out first this issue of political will I'm doing that because the first just tracks the rule of law innovation symposia we focused on political will, how to build and sustain political will and I think it might be relevant to this conversation of how do we build political will for often comprehensive e-governance or e-justice systems so Nino I heard you talk about this political will I think in many ways Georgia is a good example so there's obviously OGP that's a externality or something that helps create a political will but any other insight you can provide into how do we create political will and don't say well U.S. government or donors have to do that but not just for the donors what can other actors do OGP is an obvious example but also when we talk about externalities is the one that wants to go and the inspiration is the path that we have chosen to join the EU so that's one obvious example of how political will can be built and another thing is I really look at this simply when the government of Georgia and any government for that matter for democratic government tries the e-governance path I really would like to believe that there's no going back from that and here we talk about the simple demand and supply so it will be very difficult to explain to Georgian citizens how they can enjoy all these services that are online and how they can enjoy the services that the private sector provides and then gather 30,000 signatures on paper and submit repetitions to parliament so I would really like to believe that there is no going back and the way is only forward here any other comments on political will I think I think vision is very important in establishing will being able to show that there's a better future and I think that's part of it but I could go into more on that I think when you look you mentioned the UN Egov survey and I've spent a lot of years working on that survey and when you look at the top 10 in public service delivery because the survey also has infrastructure and education which is beyond political will sometimes to resolve but when you look at the countries that rank in the top on public service delivery almost to a T there is a strong visionary leadership from above so in the 2016 survey the UK ranked number one in the world on public service delivery and I happened to be in the UK working with the government when Tony Blair and Peter Mandelson announced that the UK was going to be a knowledge economy and they established the office of the E-Envoy which was the first kind of digital czar and it became a model for governments around the world as did the government delivery services agency GDS a kind of innovation technology driver within government and then if you look at the other top 10 countries Australia now has a digital transformation agency Canada has just launched a digital agency the United States has 18 F Denmark which is consistently high as a transformation agency so you know the leadership starts from above and that's very very important the reality becomes the way in which and I've seen it so much through the years a tool like the UN or a mechanism a really important global mechanism like OGP becomes the methodology and the community to drive forward which is OGP and then the UN tool because at the end of the day states like people are competitive and that healthy dose of competition then creates some of the political will and drives it forward but you almost never see digital government e-governance succeed from the bottom up to really be embedded across government institutions you need the political will we'll stay with you because you mentioned the the land of pilots my career I guess my question is how do we move from pilot phase into the follow on phase and in some context about scalability or scaling up where we see a development impact or a result at a more targeted level do you have any I wish I had that answer if I look at my land of pilots experience and this is prior to IBM working with the European commission and smart cities and what I saw was the tech it was really fascinating and very frustrating the technological innovations would come out of silicon valley so that by the time the 16 signatures were stamped in the right color ink and bounced back the statement of work was already outdated the bureaucracy made us implemented anyway so 3 million euro to implement something that had been antiquated because something in silicon valley had already come out that had antiquated the idea we had two years before the bureaucracy funded the project so the technology was coming out of silicon valley the government funding to actually implement it was not coming from the US it was coming from the European Union and Europe was filled with government agencies and cities especially with this but the actual policy vision to implement it was in the Middle East was in the Gulf state so everybody's heard of smart Dubai the technology would come out of the US it would be piloted in Europe and it would be embraced at the policy and governmental level in the Gulf and in MIA but where I've actually more recently I've spent a couple of last few years in Australia I'm actually seeing the Australian government being the government that is kind of I don't mean the little train in a bad way but just kind of chugging and just doing and when I got down there in the past two years they are really implementing some of the most forward looking and visionary government agency wide transformation and re-envisioning of service delivery so I don't have an answer for how you get out of the land of pilots but I do have a kind of very interesting global roadmap of silicon valley hops to Europe, Europe hops to the Gulf and Australia kind of just gets on and does it I think I always use pilots both in the rule law programming and in the courts that I managed and for me there are always ways to leverage success and prevent failure because and that's how you establish buy-in and I think if the product sells itself in other words if the judges or the attorneys see that this is going to save time and time management being one of the big issues everywhere if it saves time and you can leverage it at the pilot phase and exhibit the productivity then the expansion goals are more easily accomplished that's my experience why don't we open it up to the audience see if you have any questions for our panelists this is kind of related to the pilot question but it got me thinking in development world proposal writing and quarterly report writing and end report writing and evaluation reporting there's almost never a failure failure never happens on paper at least every project I've ever been involved in has been a smashing success and all the targets were reached and I can't imagine anybody else has a radically different experience but the the problem is that it's in a competitive environment you want to continue to get grants and you want to continue to be successful so that you can keep getting grants and one of the results of that is that our business is a little bit risk averse because you don't want to take too many risks donors don't want to take a lot of risks so they write grants and that basically have built in some sort of fail safe mechanism you're not going to do anything too risky and implementers follow that and don't do things that are too risky but when I hear about some of the discussions that we're having about technology and it seems that one of the ways that Silicon Valley has been successful in being such an engine is this whole culture of failure I've heard from so many people that you're nothing if you haven't failed a handful of times in the Bay Area so I'm curious how the panel sees that sort of mismatch can we somehow have we somehow been able to develop the ability to fail or how can we bring the two together because I think we probably need a little bit more failure let me just say that that's you more I hope I'll give you space and freedom to jump in on that one but so yeah culture of failure I will reveal that it is something that we within USAID are speaking about more and more and not to make light of it but you know instead of doing success a conference of successes we were thinking of doing a failures conference but you know there's still some residual and no one will show up but we're still giving all of our partners and implementers templates that say success story and you diligently fill them out and then we spread them around but I would just also add to it that we do have a commitment to a so-called learning agenda and we do want to look critically at what we're doing as donors in partnership with implementers and so to the extent you're writing up the reports and it's all case by case core by core I always like to see if something hasn't worked report it, explain why what did we learn from that process you know this is an area where I think there is a lot of successful movement I mean 15 years ago when I started working in this field it was think big, start small, scale fast and the motto now you'll hear more and more is think big, start small fail fast and there's more and more emphasis shift let's say in government tech it used to be waterfall projects remember all those big 10 year projects that you know the requirements were this long for the RFP and then the vendor answered it and then it inevitably failed that was called waterfall and what's being embraced now is agile that is and this is when I'm talking about we need to reinvent the way we approach things when we think about the NGO world and the funding world and the EU world that I come from we fundamentally, we don't just have to transform it so that we can have the failure conference which I desperately sign me up because I've been advocating that for years but we need to reinvent the paradigm of funding so that you're rethinking what your delivery model is and what your KPI model is because an agile model would not have you predetermine what the project is going to do and have the recipient of the aid tell you every single thing they're going to do along the way and have it locked down before the ink is drawn those 16 signatures in an agile model success is the impact you want to have and how you get there is left to be done in small sprints that iteratively learn from each other and that has permeated or filtered in, that is how Silicon Valley develops now, that is how more and more the private sector develops, government is increasingly embracing the agile development model through agencies like 18F and GDS and the digital transformation agency in Australia and I would argue that if government can embrace this model and find a way for it to work then USAID then the European Commission, then the UN will have an obligation to be thinking about that and that is about reinventing the way in which we define the paradigms for funding and the paradigms for trialing and the definitions of success you have the key note at the failure anyone else I just want to say one thing I am really concerned I have never seen a more important time for donor investment in rural and rural law and I am concerned particularly in the area of technology I think we are stepping back I do worry because the gap may be widening between the developed and the non-developed and the challenge is to help the developed obtain some of the same opportunities as the developed and I really hope that we can focus on that and continue to help Sub-Saharan Africa Latin America many of the countries in Central Asia you know some countries are going forward so fast and the economies of scale are developing so quickly in these countries and then there is everybody else and I am worried about the everybody else and I am like Pakistan they are trying so hard in Pakistan you know I hope we can continue to help that's all Other questions from the audience please My name is Paul Denage I support some projects with the Department of State Bureau of International and Law Enforcement Affairs in Western Hemisphere First off thank you to everyone for your discussion and your points from the panel I was just wondering in Latin America we see a lot of institutional changes as a result of elections what we've seen especially in the Northern Triangle is that when key partners leave we often lose momentum and as the panel was discussing there's a need for that kind of institutional buy-in from the top in order to ensure the longevity and success of these programs so my question is as we deal with these changes in the institution how can we ensure buy-in and continuity of projects so that we can maintain the momentum and contain the donor buy-in I can answer that I can try I work in all those countries you're talking about I have a memorandum of understanding with all the Supreme Courts in Latin America I found that forming a bond is critical with the country with generally the governance structure in the law enforcement, judiciary, area and ministries does change however some of the bureaucracies have leadership to identify that while there's governance transition you have to find the institutional factors that I think are consistent and that generally is at the staff level and if anything we need to do that management capacity area almost every structure has a secretariat that secretariat to me is one of the key relationship points we want to go to the Chief Justice we want to go to the Minister of Justice and they're gone tomorrow I want to establish a working relationship in Honduras with someone that I know is going to be there for 20 years in fact one of the first questions I ask whoever I'm working with how long are you going to be here and what are your goals and then we also need to build management capacity at those levels if we focus too much up here we need to find the management level find that management level and build the capacity and the knowledge and the training that we're talking about here if we can build it there I think the institutions have a better opportunity to excel and everyone wants to excel you know and our leadership though is still very important in those countries we have a great opportunity in Latin America I like working with those countries a lot that's only my thought Nina I was wondering because earlier you mentioned civil society and its role is sort of on the demand side now that the government has shown the leadership in delivering or making available different types of e-governance or e-services people come to expect it and then you also mentioned the petitions but I mean is there a particular role for civil society and the sustainability of these reforms related to e-governance we talk often in terms of social accountability and the role of civil society organizations in that accountability loop but it shouldn't always be an adversarial relationship should it are there any examples of cooperation well actually both of the tools that I presented have benefited from a lot of input from civil society especially at the design phase of these products and their input has really been instrumental because they do bring the perspective of what the citizens really want because they have day-to-day contacts through their branches with different parts of Georgian society so although some of them are also in the watchdog role as you mentioned so there's some adversity but there's a lot of cooperation because they being a part of this co-creation process also take the responsibility for the final product and that dynamic has really worked well in Georgia I think it's a good model anyone else okay I'm comfortable closing a little early so I just please join me in thanking my panelists thank you for your attention and your engagement so before we close I just want to make a couple of quick announcements first of all thank you for the panel again and the panelists and moderators throughout the day some of them traveled far and wide to be here and we really appreciate them taking time off and thinking about some of these issues hopefully this was our first but hopefully it's not the last time that we'll have a technology based discussion because as we have learned today the bar changes and moves all the time and hopefully we'll probably be having some of these same discussions in the future so we're looking forward to continuing that I want to just send out a series of thanks and I want to thank USIP for opening up this incredible facility to us and if she is still here in the room Chelsea Drear for all of the help that she has given us I also want to thank our donor State INL and JJ Welling for all the guidance and assistance that she has provided in this event as well as many others members of the Rolk team the rule of law collaborative that came up here from South Carolina Carol, JC, Hamid and Deanna for putting this together and making it run so smoothly I want to encourage everybody to go to the justtrack.org website we have a portal for interesting rule of law news we have an engagement we try to bring everybody together as a community to discuss some of these issues we have funding information and lots of other things on that website finally I want to encourage everybody to pull out this yellow piece of paper and fill this out this is really important for us speaking of data and the importance of making data driven decisions hearing your feedback and getting your comments on this will be helpful for us to improve programming in the future thank you again