 Good afternoon everybody. I'm Rob Farris. I am the research director at the Berkman Center and it is a pleasure and an honor to introduce Justice Mariano Florentino-Coyar today. He has a long and impressive resume. I suggest you take a look at it if you want to be both humbled and impressed. He is currently a seated justice at the California Supreme Court. He's spent many years as an academic. He has a PhD in political science. A law degree from Yale and an undergraduate degree from here at Harvard so he's kind of on his home turf as it were today. He served in the Obama and Clinton administrations. He's been an academic as I mentioned. He serves on the board of many foundations and has everything has done everything except I think fine time for sleep. Today he is going to talk about ankle holsters, Haiti, and machine learning. I can't wait to find out how those are linked if at all and with that I will turn over the floor to you Justice Coyar. Welcome and thank you. Thank you. Good afternoon. It's really an honor to be here. I would like to express my deep appreciation for this invitation. I've long admired the Berkman Center and the community of people it attracts. I've you know sometimes been in the audience when I'm here and I also want to give a shout out to my two kids who are sitting over here Ria and Mateo and my mother-in-law and I want to say that behind every successful man there is a surprise mother-in-law so thank you for being here Dr. Cove. When I received this invitation I had the initial impulse to maybe present a paper I was working on about cyber delegation, how it is that administrative agencies are delegating decisions to machines and what implications that might have but then I decided that one of the great things about an institution like the Berkman Center is there's a little bit more of an eclecticism both to the community that's attracted and also the kind of presentation that one can give. So that would be fun to instead take two plus years of my life and reflect on them because to this day some of what I experienced when I worked at the Treasury Department informs my work and it was certainly a pretty eclectic and interesting time and the risk of course is that I'll give this talk and you'll still think well what does that have to do with the Berkman Center why is he here what's the meaning of life but it's also possible you'll see some connections and why I feel like it's useful to juxtapose different issues so my hope is pretty simple then to tell you a little bit about these roughly two very intense years that I spent working at Treasury as a young lawyer in the heady days of the late 1990s these were heady days because it was largely a time of enormous optimism about economics and about development it was that period of time before we realized that the post-Cold War space was full of perils that we hadn't thought about very carefully and it is an interesting time from the perspective of how the US government approach this challenges financing in managing global finance and politics and also it was a time that allowed me to experience how the country survives something as divisive as the impeachment of the president for me it was just a thrill to be there I had not thought it possible to end up with a job like that right out of law school I felt very lucky I think you'll find in some of our choices today about technology and some of the trade-offs we face about governance that there are some tie-ins maybe implications for some of these dilemmas that I was involved with back at Treasury so walking into the Treasury Department my first day of work after law school I remember the lobby in Hamilton Place had the smell of green paint and new paint and I felt the same color I was arriving in the late 1990s a freshly minted Yale trained lawyer and a recently naturalized American this means I'd learned more about law preparing for my naturalization exam than I had in law school my job was helping Treasury's undersecretary in charge of enforcement to oversee five law enforcement agencies I did this while nearly everyone else at Hamilton Place fretted about the Asian financial crisis Treasury had been weathering world currency crises expanding its power relative to the State Department and managing the relationship with China mapping the structure of global interdependence and certainly not least from my perspective fighting financial crime money laundering terrorist finance and official corruption somebody you may have heard of named Larry Summers was the second in command his a CERBIC intensity fueled by a rumored 10 diet coke cans a day and Bob Rubin was the secretary and he seemed in equal parts both bond trader lawyer and aspiring rumpled professor the undersecretary who hired me was a man named Ray Kelly a once in future New York City police commissioner who was in most ways larger than life he'd also served as head of a UN team of police monitors in Haiti often describing the heartbreaking poverty in Haiti on the fringes of Port au Prince whatever the issue Kelly was confident and incisive I didn't always agree with him then or since but I found him fascinating he was just as comfortable talking about funding for UN policing as he was complaining to federal agents about some door they had kicked down by mistake since I'd grown up crossing the US Mexico border as a kid it was a special thrill to work for the man who was in charge of all those customs inspectors across the country almost every day involved a border crossing of sorts not only between domestic and transnational but also across jurisdictions and statutory schemes several weeks were consumed by the problem of new counterfeiting done by low-cost inkjet printers and by the need to kill a really really bad idea that involved requiring security chipsets on the printers that would be outdated by the time they were installed in New York I watched how Kelly still consumed with intensity about any issue involving Haiti argued with the UN Department of peacekeeping operations about a possible American role supporting the peacekeeping mission in the island nation other days the focus was on policies to limit gun smuggling and still other days found me at a White House meeting at Kelly's behest discussing treasuries role in Haiti more on this in a moment but I was also frustrated and I can tell you why with even just one example a few months after starting I wrote a memo on my own initiative to several colleagues I explained that just before a new federal gun law went into effect limiting the availability of so-called assault weapons a whole bunch of them had been sold and I mean a huge amount of these weapons because a big bulging gun sales would keep the guns in circulation for a long time the data raised a larger issue of how to consider the public's economic responses to changes in law that they knew about in advance it wasn't exactly rocket science I wrote the memo and I heard nothing for a few days then three people came into my office one after the other and warned me using a variety of different kinds of language to never write something like that again and I thought about how Congress could subpoena the document that I think about the Freedom of Information Act what might no doubt well-meaning colleagues didn't do was engage with the substance of the problem I'd raised the most memorable was a senior colleague who sounded and dressed like Vita Corleone in the opening scene of the Godfather what have I done to make you treat me so disrespectfully he asked part of me wanted to say the same thing back but I felt like a small piece of wood floating in a huge ocean that's when I decided to seek an island somewhere in that ocean where I could be a law professor someday still treasury remained absorbing even after that in part that was because different cultures coexisted so uneasily inside the department remarkably enough the Secret Service investigators and treasuring analysts that shared the department didn't always agree on the societal value of small financial fraud cases when folks on both sides got really worked up about it I'd somehow end up walking the nine blocks from main treasury to the Secret Service headquarters the agents were understandably dedicated to their work and to their views but there was another detail that was hard to forget they were armed the more they disagreed with me at the meeting the more they managed to cross their legs just right so I could see the gun strapped to their ankle holster I did my best to make sure the firearms stayed right where they were but I also learned how much depended on forging close relationships with these agents on slowing or killing bad ideas and trying to protect creativity from bureaucracy looking back on my days mediating between law enforcement agents and their vast parent organization a few ideas struck me and then struck me again they give us a framework for reconciling rules and standards pragmatism and expertise Max Weber was right that authority is wielded in different ways within organizations through charisma tradition and legal arrangements but he was even more right that organizations constitute a network of information and have authority within them and that's the vehicle through which we govern I retained some appreciation for Secret Service agents customs inspectors and the serious power of some public sector unions but I was also struck by some of the very same problems that the country later came to discover about the Secret Service I was amazed at how organizations have this ability to define a way or play down some of the core problems they face and not to face which is why the Secret Service is insular culture not only diminished its efficacy for fighting financial crime but even its wanted protection mission in this kind of setting I came to believe that political appointees with the right background could help for example Kelly created an office of professional responsibility and empowered it to gain information about the activities of all Treasury enforcement bureaus that could help and it could have helped even more if Kelly had stayed in his role but he left for a different job but it was also true that the bandwidth of even that office was limited and some of my most interesting days occurred when I was left with a measure of time and autonomy to define my own agenda or to work with Kelly and what he was most passionately determined to achieve some of those goals involved in island nation hundreds of miles from DC Haiti which then was considered more an issue than a place and implicated Treasury's financial anti-corruption and law enforcement role Kelly had a special interest in Haiti because of his role as the leader of the police monitors but a policy window also apparently opened up with both crisis driving it and also the president's interest in the region Jean Bertrand R. Steed had just been driven out of the country UN peacekeepers supported by the US had been trying to help the struggling country keep its tenuous hold on peace Prime Minister René Provault was furiously negotiating with international creditors and international financial institutions no one forgot the risk of a mass influx to Florida of conditions deteriorated further just months earlier in law school I've been representing Haitian asylum seekers now I was in the situation room in the basement of the White House representing Treasury watching with a mix of fascination and occasional dismay how even the awesome power of the US government and its military met all matter of obstacles when it came to helping to repair Haiti the weekly meetings run by the exceedingly gruff James Dobbins who later became a top official at Rand were almost always a crisp hour and 15 minutes some days the fights were about how to balance conditions with flexibility other days it would be agencies duking it out to hang on to a little glory or money but most days it was agencies trying to avoid doing anything at all about Haiti still I admired the confidence not only of Dobbins but of the State Department special Haiti coordinator David Greenley the plans that emerged sounded promising but here too some patterns emerged I saw the limits of conditional spending as a political lever in a complicated world where the very threat of withdrawing aid also diminished the funders power I found that the jurisdictional lines that divided responsibilities at Treasury between my international finance colleagues our representatives to financial institutions and my own office of enforcement complicated our anti-corruption efforts and it was only when we grappled with that reality that we were able to devise what sounded to me at the time like a compelling plan to help Haiti become less like Haiti over the next decade fast forward to January 12th 2010 when a massive earthquake hit Haiti many years later at 4.53 p.m. that day the country was devastated by the quake centered about 16 miles from Port-au-Prince credible figures suggest the death toll was roughly 100,000 or more about 3 million people were affected the cholera outbreak in October of that year compounded the tragedy by then I was back in the federal government this time at the White House and again attending meetings about Haiti the U.S. government deployed the military and changes followed in the mechanism of our relationship with Haiti funds and supplied flowed to Port-au-Prince we granted temporary protected status to Haitians who were in the U.S. but still the human toll was staggering as was the reality that Haiti wasn't a particularly well-functioning place despite all the effort that my Treasury colleagues and I expended back in the late 1990s eventually the spotlight of global attention drifted away from Haiti meanwhile per capita GDP and Haiti today is almost $100 lower in real the constant dollars than it was when I attended the NSC meetings about Haiti in the late 1990s so I came back to see some of the risks and contradictions of our policies on Haiti back in the late 1990s I was rushing ahead on another front that seemed promising at the time for whatever reason I was drawn more than my other enforcement colleagues to the banking regulatory and economic policy aspects of Treasury the economists were doing traditional econometrics for the most part and estimates about macroeconomic policy but it was the bankers and finance folks who were most interesting to me the best of them were huddling in the hallways beginning to imagine how someone might design a system to mine entirely online currencies even as the lawyers occasionally rolled their eyes more conventional electronic money raised interesting questions about the future central banking and about what kind of company might eventually get rich capturing the new industry they were also becoming increasingly aware of the havoc that algorithms could wreak by magnifying trends and creating aggregate torque like effects that might have outcomes that were not predicted or desired I thought back to the limited experience I'd had attending school in Texas using rudimentary computers than available and it wasn't long before I was asking hard questions about what we did with all the data in Treasury computers in Vienna Virginia Troves of anti-money laundering data sat in servers in Virginia currency transaction reports suspicious activity reports case files a few months into my job on a trip to one of these data centers I was struck that by the insight that drove my first and way too long law review article years later in fighting financial crime we often talked about following the money as if our data allowed us to do that but in fact we were detecting most financial crimes in very ordinary ways through undercover operations or informants and then using our data to make it easier to impose penalties on people already detected though I'd grown up in places where drug related crimes sometimes got ugly drug laundering is not what motivated me though it was on my mind I was instead motivated by the prospect of doing something about international corruption I kept pressing our agents and analysts if we could build the software and hardware infrastructure to analyze millions of currency transaction reports suspicious activity reports and other forms of information and if the algorithm could learn from agents and from its own success wouldn't we be able to change the way corruption investigations worked around the world in a few fateful conversations after I'd started really pushing this agenda I came to see how complicated and full of trade-offs this agenda really was pre-911 it was easier for other officials who didn't see it the way I did to push back when memorable conversation happened with an extremely bright sec lawyer I think a Harvard grad actually who accused me of trying to build a legal infrastructure that would facilitate pervasive cross border government surveillance do you really think subjective judgments by inspectors on the border the ones who racially profile are better than what we could get with the right machine I asked back in one of these heated conversations thinking about what it felt like to be at the mercy of a human inspector all those years as I was crossing the border in Mexico we agreed to disagree more than once but I came to realize in the months that followed that the question was really hard how do you design the algorithm how do you overcome what were then not only legal constraints but big limitations on processing power to achieve the goals that I share with some of my allies what to do with financial crime investigators of the machines began to prove better than they were could we get the in-house knowledge to aggregate different data sets on different platforms or maybe somebody in California could start a company to do that the thought crossed my mind to go do that but being a law professor sounded like a lot more fun I think I made the right choice my wife might disagree we made some modest progress in the years to come steps that accelerated I later learned post 911 in the end I gave in the allure of academia I wanted colleagues like the colleagues that people have at the Berkman Center and practical constraints combined with the culture of Treasury enforcement to slow down the pace of the reforms that I was advocating the fact is that Treasuries law enforcement agencies were still largely run by folks with ankle holsters and they weren't so keen on a world if it could be built where they were spending quite so much time working alongside machines they didn't fully understand nor was it easy to actually build the systems that we were talking about network computers were massively slower than and few insights from AI and modern interfaces were available Bayesian inference neural networks Palantir style data visualization none of that had made its way to Treasury enforcement at the time finally there was a law itself finding ways to do routine analysis of wire transfer data in real time was not so simple from a legal perspective and barriers that later got a huge amount of attention post 911 and actually collapsed post 911 were a big problem back then by the time the Patriot Act passed in the early 2000s I'd come to see this domain in a much more nuanced light and to see my Treasury project as part of a much larger issue facing humanity as processing power got cheaper and better and more accurate data became more easily available I'd come to see that the SEC lawyer was right even if I had been to much has indeed changed in California we have Kim Chippewa Govittacos sideburns are coming back and perhaps most significantly there's a bit less of that heady sense that American power can reshape the globe but it doesn't take much to see how much of what I saw a Treasury then still is with us we depend on organizations and institutions to solve our thorneous problems to make a smarter and saner in a sense to help us resolve our disputes and to be our best selves as we struggle with our limitations as humans but as Max Weber would also admit we're rarely able to engineer away the humanity from our organizations and it's a pretty ugly thing when we do ugly is also a good word for the governance problems that persist with all respect to the nuances that come from the study of region or culture you could cross out Haiti and at Afghanistan from my discussion above and see how much our governance and development plans go badly awry and maybe even more pressing of the choices that we're going to be facing about machine learning and about surveillance what felt like more distant vistas 18 years ago at Treasury are now just on the horizon if not literally upon us we have staggering amounts of data and computing power interfaces that are studying that at pretty good at communicating with humans and the failures of organizations and governance we haven't figured out how to solve will no doubt encourage at least some among us to argue even more forcefully in favor of turning over more of our lives to complex algorithms helping us optimize water use energy health care traffic and love that's the world some will love and others may love it may be a world not so different from our own or a blissful place of leisure and creativity for billions what's more likely I suspect is that we're heading to a world of possibly not only certain people but many many kinds of people and many kinds of job categories losing their jobs and a whole lot of others who are going to be living with very little understanding of the algorithms driving the rhythms of their lives money laundering enforcement maybe easier but the people ostensibly protected maybe living with new forms of inequality at best and worse even dreaming up the job descriptions for my kids is getting more difficult all of which makes me wonder about what some young lawyer joining Treasury today is thinking about and what kind of world will leave our kids as that Treasury lawyer figure figures out her career in the years to come. Thank you very much. Who wants to start us off? It was promised a whole bunch of questions. Okay, you mentioned somewhat briefly that Freedom of Information Act because they call FOI. Can you explain to me your opinion about how this functions? I don't think it does function that's my opinion. Yeah so like I mean let me channel a little bit the theme which is what it felt like to get a FOIA request when I was a Treasury. I mean it was one thing to be in law school and to take Jerry Michal's administrative law class where I thought FOIA was a big deal. I thought FOIA was great and important. I worried about how broad the exemptions were and I imagined when I was taking administrative law that there were these wonderful completely dedicated people in government maybe a huge office of lawyers for every agency that did nothing but respond to FOIA requests and press each other to make sure that their responses were as thorough as possible. What I instead experienced in government in the early days of email traffic in government was having a ton of stuff on my desk being pulled in a hundred different directions and then getting some email like at 11 p.m. on a Thursday saying we have a deadline please produce all documents that are relevant to the following query you know gun investigation comma ATF New Hampshire whatever and you know I would do the best that I could but I guess I to this day I sort of still feel odd that there isn't that cadre of lawyers that I imagined and that in fact the burden falls in a disaggregated way that dilutes responsibility and a whole bunch of people so I'm sure that we can do better. Hi I'm Matt and I'm at the Berkman Center and so I think the question that I have is since it seems like you're thinking about a lot of this machine learning algorithm stuff early what kinds of things did you feel like the people who are impacted by the machine learning algorithm should understand or know kind of as they deal with the process especially I mean I think one of the tensions you deal with is oh there's privacy but there's also the government's need to enforce so just wondering where your thoughts are there. I think they should know that deep inside almost every algorithm is a whole bunch of trade-offs that somebody decided to make a particular way and that whether those are visible to you when you consume information that is provided to you is highly contingent meaning it's not at all clear that those trade-offs are visible so there are at least two ways that that can be problematic to me one is that the designer of the algorithm wants to make sure that somebody consuming the information says well okay this person appears to suspicious but it's kind of 60-40 and it's a close call and somehow a separate team designing the interface how the information is displayed or how the voice sounds when you hear it in real time doesn't capture that. The other problem is fundamentally the designer of the algorithm may not be as familiar with or is interested in what the trade-offs imply as the person who's closer to the on-the-ground reality and that separation between the designers over here and the people who are going to consume the information over there is to me a real issue so let me just play it out in the law enforcement context is one example of where this might come up. The reason to think that we could do better than just somebody making a judgment call on the street deciding whether there's probable cause or not is because we know a great deal about the limitations of the human mind and I would not cast that aside and ignore that I think that's really important that's why we have discussions about implicit bias among other things but what happens in that translation of the knowledge that people have about what constitutes suspicion to a set of choices about how you code and what gets lost in that translation and particularly given that sometimes counterbalancing those biases is experience or tacit knowledge or something like that how do you create a community that can deliberate and not lose the ability to sort of learn not only individually but across organizations when a lot of choices outsourced to a different technology. Yeah I guess kind of the other part of that should the algorithms be open should we be able to FOI the algorithm and audit it or do they need to be hidden in private so that they effectively work so people don't cheat them? I think it's it's really tricky to try to live in a world where there isn't some accountability at the same time the obvious dilemmas which I trust the Berkman Center to help figure out that they're both on the kind of IP and trade secret side but they're also on the government operations and effectiveness side so you can't necessarily be completely transparent with all the information that law enforcement is using to decide what the targets and if you want people to develop certain technologies you can't necessarily force them to open it all up so that implies lots of conversations about both administrative law and about open source and other things. Can we just pause on that topic for a minute I want to follow up so you've been inside the sausage factory trying to figure out how to apply law. So that's what the smell was. I actually love sausage I say that but given the choices I know it's a false choice but given algorithms which are transparent but are very deterministic versus rules and guidelines that offer a lot of discretionary authority to individuals to implement in different ways how do we choose between those and what would even the criteria for choosing be. Yeah that yes is the answer I well let me resist the framing of the question for a moment and this is not me just trying to to wrestle out of a hard choice but but it's more suggesting that I don't think we're at a point where we can choose I think we're at a point where we might most responsibly try to create parallel structures that can learn from each other in other words let's let's think about some of the choices any regulator has to make how to use very scarce resources to enforce laws of different kinds whether they're consumer protection banking laws environmental protection laws a discretionary regime relying on some combination of judgment discretionary judgment human judgment traditional forms of analysis is not irrelevant yet and shouldn't be for all kinds of reasons including that we haven't yet captured the full value of how the mind works through problems or how organizations do but nor should we necessarily completely limit our interest in trying to see how those judgments might be well informed by different kinds of technology that might have a kind of an audit trail to them generated about how the decision precisely was taken but let me then make two further observations one is that not all computer systems work that way so I think there's an important distinction between neural networks for example and conventional heuristic algorithms because you can't really tell necessarily how a neural network at a very detailed level has causally you know decided to optimize this way and then the other is I would resist the notion that algorithms are by their nature transparent so let me see if I can give you an example let's compare earmarks to other forms of legislation so my former colleague Larry Lasse whom I guess I'm over losing at Stanford but no is now here targets the problems that arise with earmarks but there's one thing that you can say about earmarks that at least maybe is a little bit of a countervailing force and that is that you can make a pretty easy argument that something is fishy in an earmark because it's pretty intuitive to explain to somebody often look you know this particular lawmaker did this particular deal this amount of money is going to a particular interest in that lawmakers district and isn't that corrupt some highly technical change in the occupational safety and health act that changes something about the concentration of chemicals that counts as being toxic in a workplace is not nearly as obvious because it's very very complicated it's not to say some interest group can't or you know stakeholder can't make some argument that there's a problem there so what I worry about with algorithms is that it may be in plain sight but the complexity of exactly how the trade-off has been made is not something it's easier intuitive to explain to the public hi I'm Sharon my question is Sharon are you connected to the Berkman Center I actually work with a project that was kind of supported by Berkman Center so my question is you're talking about biases and trade-offs and the the basis of my question is what about new biases that will be formed through these algorithms I mean humans are the ones programming these algorithms but through interactions amongst many different algorithms is it possible that new biases that we don't even we can't even fathom yet are formed and in that case how do we is that where human judgment comes in and how do we prevent that and then how do we also I guess eventually we'll want to teach a neural an artificial in AI network what a what a bias is and how would we go about then putting that into a an algorithm how do you I guess teach AI what bias is that's terrific question I think we're learning about how best to do that by better understanding how to distinguish those features in human cognition that are bad and that we think uniformly you can make a case that we'd rather be able if we could to engineer our minds not to have them and those things that to some statisticians or economists might seem like you know lack of of rational reasoning but you could actually make an argument that are sort of part of human nature and I think that's the prerequisite to then thinking about how we use computer systems to either get us away from bias or to preserve the germ of human thinking so that we don't get off track so let's take one example that has been written about a lot in the context of regulation and that's nuclear power so if I'm very drawn to conventional forms of cost benefit analysis I can make a good case that the way human beings think about the merits of nuclear power is biased because they overweigh the risk statistically speaking they overestimate this because of all the things that professors like to write about involving availability heuristics and graphic nature the information you remember but does that really mean that that's a bias and a mistake that we'd rather engineer out of our thinking I guess I would think I sort of lean slightly in the direction of yeah it's probably a bias but I'm not sure that the logic that leads people to draw that conclusion is necessarily so flawed that I'd want our computers to completely engineer it out maybe it's something in evolution that's a failsafe against a particular kind of thinking or maybe it's just who we choose to be or maybe at some point we have to ask about democracy and human freedom so that's sort of one way to think about it the other part of your question which I'm trying to think about a lot when I'm not thinking about my day job is the different ways that computer systems can perform in a manner that is not what we intended or hoped for and you could think of that as a bias so I'll mention a couple scenarios so one is you could have emergence problems where you have algorithms that are designed in very sensible ways to optimize vis-a-vis the interest of the person who's using it but when they interact so think about flash crashes for example where lots and lots of folks are using algorithms to trade and they respond to each other in ways that were difficult to model even if in a particular set of assumptions that we're gonna work fine there's that as a problem then there's if you use neural networks or more Bayesian approaches the training data that you feed the system you know if that's biased you know if you have a distribution of decisions about suspicion made by police that had themselves some bias that you can't easily see that will begin to get built into the system and then third is expertise right the folks who have the expertise in developing algorithms are not necessarily the ones who know about money laundering or about education or about you know biological chemistry so they might think they've understood the essential concept but not necessarily captured what the expert knows Paul Cosway from Boston University I wonder if you can give some comments on what you based on what you saw the Treasury in the 90s and what you see today around cryptocurrencies electronic money Bitcoin and I'm a little less interested in the anonymous money laundering off the table payments and more as that might roll into legitimate payment systems it's not just the transaction but all the the record keeping around that how do you see that proceeding from here thank you for the question I'm gonna think out loud a little bit but I'm gonna draw some very general paint with broad brushstrokes I think that my colleagues at Treasury were correct to recognize that this was coming they saw it before it really hit I and at the time they ultimately felt that the right way to deal with it was highly incrementally in other words not to try to come up with some broad general framework that was meant to regulate this and that that's not surprising given the the tenor of the Treasury Department but also everything wasn't known thinking now a little bit speculatively I suspect that my colleagues my colleagues and I failed to fully understand how much disaggregation could occur in this space I think it was harder for an institution like Treasury that was designed in a way to deal with private sector actors attended to concentrate a lot of power and money and tend to be big like even the small ones are big right to fully understand how a culture of startups and rapid innovation and cross-border developments might change this I think it another way to say that maybe is I don't remember a lot of thought being given the mobile payments technology it was more a sense that cards might be different but they wouldn't be that smart they might have a chip right but the notion that you would actually be engaging in a whole bunch of transactions on a supercomputer that fits in your pocket was not really on the horizon but and you know looking back on it I think of that level of understanding and prediction as being both a case for what's achievable by a government agency with a lot of blind spots but also there were a lot of costs involved in pushing back on ideas that were not good ones not precisely on this payment stuff but on the enforcement side the story I alluded to involving the color printers the oh my gosh you would be you'd find it really interesting just how much time that took it took a lot of time well that taught me more than anything that part of serving in government is slowing down raising questions about or trying to shut down bad ideas and that you can sort of really feel good about doing that even if you couldn't point to things that you say like oh that was a great thing that happened and it happened because some people and I worked together on it it was really hard to get people to understand it first and then they did eventually just how fast the pace of innovation was and what the implications were of the razor thin margins on those machines but once that was understood then it became clear that we weren't going to go in that direction anybody want to follow up thank you Kevin Cullinan vaguely associated with Berkman following up on that question there was a an article in foreign affairs recently about the US use of its economic might and and its alienation of former partners in its use of the treasury tools and others would you comment on how much the cryptocurrency or digital currency capability will will facilitate the creation of alternative international settlements I I'm going to punt on that one because I don't know I think there is a possibility but I I there are definitely people in this university more thoughtful than I am on that particular issue but I do think I can speak to one aspect of your question in other words just to play that a little bit more it was it was hard for Treasury to predict at the time in my recollection at least that there'd be this level of innovation and payment systems at the consumer level and even though right now the conventional wisdom is you know central banking a central banking in the reserve currencies in the world and so on I think it it's not unreasonable to think that there are a whole bunch of different scenarios that could develop and whatnot notwithstanding the fact that summer seem more likely to us now and are probably more likely as a practical matter that said I think the the point you're making brings up to me one of the major tensions in the interagency process which is again something you don't learn about enough in law school I certainly didn't learn about it much at Yale and I found that a huge amount of my job probably thirty five to forty percent was consumed representing Treasury's interest in interagency discussions now my initial reaction to that and it relates to your question I'll show you why my initial reaction that was incredible frustration because I felt like that was a highly bureaucratic thing and it meant that when we had some idea for some new way of doing something that we had to go through a process that involved a whole bunch of other players and it wasn't clear if they were veto players or not I began to feel differently about it when I realized that the flip side of that very coin was that for any decision that plausibly affected Treasury even in a small way we'd be in room and we'd at least at least get to shape it and one of the axes of debate between Treasury and the State Department often was about what the diplomatic consequences might be of something that seemed very well thought out from a Treasury perspective near the tail end of my time there I began to see a shift in the direction of those conversations that I thought were interesting where there were times when maybe the State Department might be pressing Treasury to consider doing something and it was Treasury concerned about the diplomatic implications but in a way from a kind of financial diplomacy perspective so it took more the form of like well we have these partners we have all kinds of interdependence with them and maybe we shouldn't go in that direction so I found at the time the Treasury was often the place where things got slowed down on that dimension that probably shifted post 9-11. Hi Vivek Krishnamurthy with the Cyber Law Clinic at Brookman. I want to shift back to algorithms if I can and thinking about the regulation of the use of algorithmic inputs into government decision-making of various kinds do you think that there could be a point in time where we would be comfortable enough with trusting the algorithm as decision-makers such that in your day job as someone who reviews government decision-making you can say yep you know you relied on this thing that's good enough or do you think that democracy and the rule of law and other kinds of fundamental values that we have dictate that a human ultimately has to be the person or the decision-making entity a human can rely on it but it's not enough to say that these five data points are probable cause right it's these five data points with some kind of black box human analysis as a backstop making that call. So there's a line of cases in federal administrative law involving something called the presumption of regularity which is it may seem a little esoteric but it's the idea that if a government agency makes a decision those that agency has to follow certain procedures in making the decision we presume for example that if the law says that the undersecretary of agriculture has to approve something absent some incredibly graphic reason to think otherwise we start from the premise for discovery purposes another thing from us of assuming that yes it was the undersecretary of agriculture that truly made the decision that's the presumption of regularity that is the kind of doctrine that raises for me kind of an antecedent question to yours which is let's suppose that there is a state of the world out there where we could imagine designing a machine that's so good at doing what we'd want it to do to follow the law to maximize social welfare while it follows the law and to tell us a beautiful story about how it did that and to do it with a beautiful tone of voice and to smile at us all at the same time well before we get to quite that point we might have hard questions that we're going to be facing about situations where we haven't made a social decision that we're going to delegate to the machine we have however made a decision that is perfectly fine to inform our judgment with one and then we have these legal doctrines like the presumption of regularity that create presumptions that the decision occurred the right way right so in situations like that my concern is going to be how do we how do we regulate the level of reliance of somebody who says to you at the beginning of their day I'm here to make sure that the buck stops with me I'm a decision maker the decision is mine of course I'm going to inform myself with tools that I have or my staff will have and when they come in and talk to me they'll tell me where the data are coming from I guess I worry that we haven't yet figured out how to strike that balance given what we know about how people respond to human-computer interfaces and furthermore I also think it's an interesting problem that fortunately people are beginning to think about about how courts then play this role right what is a court want to see to try to feel like the decision was taken the right way so in a way I'm just restating your question and embracing it yeah or or you know let let me throw out another idea you know again in the context of things that have not come before me right and whatnot but Elena Kagan and David Barron both of whom are now in my profession I might add wrote an article once talking about how Chevron deference might be appropriate and for those of you who don't like love administrative law and spend like all your days reading about this the idea that that you might give an agency special deference when you consider what decision that they've made about it how to interpret a statute that you might apportion that Chevron deference on the basis of how much a senior level administrator truly was involved in the decision which is kind of an interesting argument I'll leave it at that Michel Raymond from the University of Geneva bouncing on that question from another angle do you think the legal community at large right now is mature enough to be made to be able to make judgment calls on these sort of issues talking about algorithm etc in a way like legal discipline is being taught how people are selected to go into courts I refer for example for the recent lands versus universal judgment of the Court of Appeals of the Ninth Circuit where they basically said okay the YouTube algorithm that determines if there's a copyright violation it broadly complies with your process and we won't even think about it because we're not really technicians I think we're early in the legal conversation let's put it that way and I would love to see more attention on the part of judges and legislators just to the dilemmas and the questions more attention to the kinds of issues we're discussing right now but I think that part of what makes it hard to draw a connection between simply more technical knowledge and better outcomes is that the dual nature of expertise as a as a societal good and here one person who made this point nicely is Steve Breyer who wrote it up in a book called the Breaking the Vicious Circle and it's a pretty short book in fact it's a book shorter than many law review articles which is something we should all learn from probably certainly I should and the his point was that we have set up the administrative state in part to leverage expert judgment and expert judgment could include more familiarity with how algorithms work and so on but the very same things that make expertise important and useful can give people tunnel vision and create among them the illusion that they have mapped out the full range of relevant considerations so in some respects the irony here is there may be a curve between knowledge and societally useful outcomes that is not linear and monotonic but that actually is sort of like a like an N shaped curve I'm making this up I don't know what what the shape of the curve is exactly but I could imagine an argument that there's an optimal amount of familiarity so you spot the issues more but you don't actually have so much expertise that you have confidence that actually it's really not that hard to program an algorithm to take account of all the different considerations and now we're done so you know how does that translate to the law school curriculum well I'm all for embracing the idea that lawyers should learn more about coding and about algorithms and about neural networks and that should be a great step forward for the most part but I don't think that coder lawyers are necessarily going to solve these problems because of their expertise because I think part of what is required is a dialogue and attention between people who believe that the technology can do more than it currently does and those that come at the problem with some serious skepticism about it. Thank you so much for your presentation I just want to take you back to Haiti because your title implies Haiti here and that connection between diplomatic decisions and Treasury Department with some probably corrupted governments around the world but in the case of Haiti will you have any story about anything that you dealt with involving any corruption in Haiti that you want to share with us? I think I can't really get into that too much and and the reality was at the level that I was working on I was not literally seeing cases come forward about it but I was dealing with the frustration of a government in the US that was trying to find a way to deal with the government that it could only partially trust and I guess what I would just reflect on right now is I think that this is a real dilemma for countries that are trying to affect the behavior of other countries with aid at the end of the day because the very act of withdrawing it which is what you're supposed to try to do to achieve the result you know it's not always completely credible because of the alignment of interests and what is it going to do to your influence going forward and I don't I don't think we've been perfectly honest about the limitations in our ability to do this I think that the amount of collective action across countries to deal with corruption that is required is more extensive than what we currently have but I do think that when we think about a domain like money laundering enforcement it's important to recognize that even if anti-money laundering enforcement has not eliminated all corruption it's actually a pretty relevant tool here and to the extent that there are civil liberties concerns and I think those are real on the one side here I think it's worth using the example of corruption as a different domain of where you might get some leverage and traction and how it is that given the failures inside a particular jurisdiction you may need to rely on what you can detect from the transnational banking infrastructure so I think there's more work to do in that score and I don't think we were wrong to think of that as one piece of the puzzle that could be developed Peter Hurdle of Berkman affiliate you've been smiling throughout your talk all day today and I'm not hearing a lot that makes me feel very optimistic the so let's think about Haiti in particular where we heard that there's interagency conflicts that we end up things have only gotten worse that it's the same with Afghanistan give me a reason to feel optimistic that the government can actually do something to make things better and move us forward so if you look at a 35 to 50 year period I think global hunger disease has fallen fairly dramatically we educate 80 percent more women or I think the way to say this correctly is in developing countries 80 percent of girls now finish secondary education this was not stuff Treasury was working on but my point is I think that human practice is a real thing and particularly with respect to health and education it's taken substantially forward and I think some of the reasons why that's not so salient to us is it doesn't get covered and it doesn't get mentioned quite as often so I'm smiling because I like being here with you guys I I think in in some ways the conclusion that I draw from those two years in government are more mixed I I feel like there are great many reasons to feel like organizations naturally have blind spots like in particular our efforts to support and promote governance and development and limit corruption in certain countries are much more fraught than we think and I feel like the dilemmas facing us involving our reliance on technology that a lot of the consumers of it don't fully understand our big and and you know substantial in all kinds of ways but I'm an optimist about a couple key things I mean for one I think that inequality is one thing but human welfare as measured by health and nutrition is likely to continue to improve in a fairly straightforward way I feel like by and large I'm optimistic about the direction of this country and what it can do in the world I feel it's an indispensable nation autobiographically I feel like a little bit of my optimistic nature is rooted in the fact that I was born in a different country and I came here so I I see the limitations but I certainly see an incredible amount of that we've achieved in this country that often we take for granted I don't know if you have like an opinion on this but I want to make like comment you're talking about came in the things like 80s one of the presentation and this is a country that has very limited natural resources very little to offer except for human capital they are the fastest growing recruits in the Boston Police Department that they drive most of the cab drivers are Haitian guys for extremely you know valuable part of the human capital is the biggest can you comment on that well take that as an opportunity to just reflect on something that plays out in our in our heritage as a country which is the commitment this country has had over many decades and in ways that can sometimes be divisive to including people as immigrants in our society yeah that's what I wanted well I all I can say I think it makes the most sense for me to stay pretty general about this is that I think that we need to both appreciate the subtle ways in which we integrate people into our society for immigrants but also recognize that that you know changes over time that it might be different in different regions and for different groups of people but I do think it's impossible to have an intelligent discussion about what this country is about without recognizing its sort of long-term relationship to immigration policy and the ways in which that affects every single community country just on your feet for a long time is that we collect last several questions sounds good handle all of them what do we have out there I so I have I have one question to slip in first is the moderator prerogative so a little bit going back to the question of optimism and pessimism so you've talked a lot of different ideas in this talk and also given us the two decade perspective we don't often take it when we do we're thinking about pets.com and Netscape but leaning into the institutional and organizational roles what what are the topics and sectors that you feel like we're making good progress on that we collectively are better positioned to deal with as a as a society and which areas might we be slipping on or not as well equipped to tackle moving forward. I'll leave you to think on that one while we collect a few more. Somebody else had a question. Hi my name is Holly and I'm a perspective master student here I'm actually visiting from Nicaragua and I work at an NGO there so I have a different perspective perhaps on this but I wanted to ask back to your point that education and health have been steadily improving. I'm studying international development and working in that and seeing it in the field in Nicaragua and I don't see across the board in developing countries data that that's actually true. We failed on all the MDGs we accomplished partially some of them and we just set 17 more that are somehow supposedly more sustainable but as far as education and health we've been stalling we made some good advances but then the easy targets were met and then the rest were difficult. So I'm curious to see what evidence you have of that being the case. So I'd suggest Angus Deaton the Great Escape and you know the World Bank and the UN keep data on this and I think it's fair to argue with some of the measures they use but I'm mostly relying on Angus Deaton and I think the reason why it's possible for him to be right in for you to be right at the same time is a question of time scale. I think if you ask if you ask whether most developing countries this is at least my understanding of it and you might have a different date or different perspective make progress in their development goals over a 10 or 15 or 20 year period. I don't think you can make generalizations about that because of some of the governance problems we've been talking about. If on the other hand you're looking at aggregate figures life expectancy for example over 30 40 50 years if you're looking at you know global hunger I think the trends are pretty clear and I would note that to me a reason to be concerned about trying to understand what the full implications are at a human level is the distribution question. You know averages can tell you something valuable but they can also and something valuable you can feel good about but they can also mask a lot of inequity and ultimately I think the next level of depth in the discussion for me is to ask well why is it that some societies that have similar resources are actually seeing such incredibly divergent outcomes so take you know Mexico and Russia for example just to pick two where the life expectancy change has gone in completely opposite directions so I I don't mean this to sound polyannish and to suggest that simply because over 40 50 years there are some trends that look good that that means we should declare victory I think that you're right to bring up some of these concerns and to your point I I think ultimately what what I would celebrate from a law and policy perspective as we think about how we administer ourselves and how we govern is how much progress do we make over time in identifying tools that help us better diagnose what's happening in our country who's being left out you know what failures are due process look like in my own role now my role is quite limited it's about resolving disputes and about keeping the focus on the case before you but I also get involved a bit in how the courts in California which are the largest legal system in America govern themselves and how we provide access to justice and I would say let me just use one particular issue as a metaphor for the kinds of technical and governance challenges that we still face and deserve a lot of our attention so if you add up the people who don't speak English very well who are limited English proficient in the state of Florida and the state of New York and the state of Texas and combine them that's about how many limited English speakers exist in California we're talking about seven million people 210 different languages and the question of how they navigate a court system with millions of cases and thousands of judges is not an easy question I'm very happy to see that in the last few years we've been making steady progress in California providing more interpreters to people where appropriate looking at video remote interpreting technology looking at how else we might leverage technology looking at how we might leverage human capital but ultimately the reality remains we're talking about an incredibly diverse population that has to live together in one jurisdiction that needs and wants different things at different times and I think ultimately for all the interesting things that our technology can do I worry about coming too quickly to the conclusion that it can solve all the problems that arise and all the opportunities that arise from that incredible mosaic of diversity and if it looks like those problems are being solved I would interrogate the point and sometimes wonder whether something is being elided okay so to stay on this issue of the optimistic a prognosis over the I guess the medium or long term you didn't mention climate change once in all of your remarks and I wonder how you feel climate change the dislocations and the massive destabilizations that are going to be produced factor into your optimism I think it's an area of great concern and the only reason I don't say more about it is because cases come before me about that kind of thing cases come before me about that so but you know I I would just note that it's hard to have a discussion about everything going on in the world without recognizing that this is a huge challenge that humanity faces