 Good afternoon everybody Thank you for being here. My name is Andy rich. I'm the head of the Truman Scholarship Foundation, and I Appreciate you coming out this afternoon. It we're all sort of I think bleary-eyed following the election and Spend an awful lot of it last night and this morning kind of following Election results and so it's to your credit that you're here as we switch topics in some sense But but it's the topic to which I think many of us now Turned quickly which is a topic of governance now with the elections done We look back at how well this town can make decisions And search for better systems for governance or better methods of governance We've got a lot of problems in Washington and in Congress in particular we've got institutional problems. We've got electoral problems We have political problems, but we we also have problems that We talk about often with respect to information about the quality of information both the Abundance of information the purpose of information and how effectively information and expertise actually reaches decision-making or is useful in the governance process and and so that is what We're here to discuss today for a panel entitled information overload how elected leaders find expert knowledge in the internet era And we're gonna hear from three folks who are Not just thinking about these subjects, but are actually doing something to try to improve Not just the quality of information and expertise that is available to decision-making into governance But also to improve the systems by which expertise and experts themselves reach decision makers So we're gonna hear and we'll hear in order from Gary van landingham Who's the director of a project called results first at the Pew Center on the States from Lorelai Kelly? Who is the leader of the project here at the new America Foundation called smart Congress and from Ben Freakley? Who is a special advisor to the president the school of public affairs at Arizona State? And we'll talk about a project that they have launched called decision theater And so if you don't mind first off if everybody can turn off their electronic devices those that make noise at least And prepare to be engaged in the discussion where each of them is going to speak for five seven minutes And then we want to open it up for your questions comments and really get it into a conversation and so With that I'm going to turn it over to Gary Thank you, and I welcome the opportunity to to come and share the work that the Pew Center is doing on this issue and I love that the topic that we're talking about information overload because I Think that we have to recognize that we are living in a knowledge In a information-rich, but knowledge poor environment meaning that there's a lot of facts out there But I'm someone who actually believes that that facts exist and we're not totally in a postmodern world but but the question is With all of this with all that information with all that stuff out there How do we really package that and really find the nuggets in all of the The ore that's out there and make that information available to policy makers and to do so in a way That is actually useful to them and hopefully compelling to them And I think we have to recognize that there are there is a lot of information out there You know if you walk the halls of Congress if you walk the halls when you legislature You're not seeing a lot of members sitting around saying there's a problem here And I can't find anybody who would tell me what I should do about it I mean that there are thousands of folks just one block over on K Street who make a good living You know providing information to policy makers the question is we recognize that a lot of that information isn't particularly good There's a lot of research out there that isn't particularly good But how do we find and package the information that is really good that really can help inform the policy debates and to Really help us move forward in a way that that captures the knowledge that we have in a way that really helps us make better decisions So I think that really is that the challenge we have in front of us We certainly face daunting challenges as a nation as his states my work focuses on state governments We all know about that the federal budget deficit that that are out there states under the Great Recession have Faced a equally daunting set of challenges since the Great Recession ended States have cut about half of a trillion dollars out of their budgets, and we consider the size of state budgets Collectively half a trillion dollars is a lot of money from from states my home state of Florida is now around 22% lower in revenues coming in for the general fund than it was in 2006 that sort of shows the depths of the challenges that states are facing So we the question for us is we can either sort of try to cut our way out of this or to try to make better Choices to to to get our way out of this and I think that we really have an opportunity to make better choices Not necessarily just to to cut for the sake of cutting That the challenge with cutting things is policy makers don't have good information on what what programs they're actually operating No state has a comprehensive list of all the programs that are operating They don't have information on how well all those programs are operating So a rational choice for a policy maker is to cut across the board and hope maybe the agencies can figure it out And can cut stuff that needs to be cut because they simply don't know it Well, the challenge with that is we end up in a situation where we cut everything and nothing works because we're so under-resourced Everything that nothing can possibly function the way we want it to and we did the challenges face in the country are too severe to Be able to to cope with that we really start have to need to start making better triage decisions Identifying what really works out there putting our resources towards that and then you're stopping doing things that don't work Now that sounds great. The challenge of course is what are those things that work? Well, and what are the things that don't work? Well, and you know How can I really help make those choices more compelling to policy makers? And the good news situation that we're in is that a huge amount of really good research has been done across this country And in other nations really over the last 20 or 30 years a lot of really great evaluations have been done that have used very strong research designs to Identify collectively sort of what we think what we know actually various different types of programs can accomplish and For example certain programs saying criminal justice have been Evaluated extensively all across the country drug courts have been evaluated in almost every state different sensing options are Well researched a lot of educational programs are well researched So if you take all that information and you can put it all together Collectively we actually know quite a bit about what works the challenge for policy makers is they don't have time to weed through all that information So what we are doing results first is doing that for them and and making it very readily accessible to them our research partners at This point have read through well over 27,000 evaluations of Casting aside probably 90% of them which just aren't really good research designs But then have used that you know statistical analysis techniques to say based on our collective knowledge of what we know Here's what we expect a program like this to be able to accomplish if implemented correctly in a state and then We go the next step and say based on your state Your state's characteristics the people that you're serving the cost of providing these services in your state We can not only give them information on what the policy choices will be But also a very comprehensive cost-benefit analysis of each different program option that's available to them What this really allows us to do is what we call sort of a consumer's guide to governance saying that Yeah, you want to do something about crime in your in your communities Here is a menu of proven interventions, which would be effective Based on all the best research if you did them in your state And here is a precise estimate of what would happen if you implemented each of these and you can compare the results of them And the great news is we can do that now We can do this in a way that it has never been able to done before and it's something that really is compelling to policy makers Because it allows them to say on an apples to apples comparison if I do this option Here's where I will get if I go down this road Here's where I will be and here's the comparative return on investment that I will get from different options the same type of Analysis that we all ideally want to be able to do when investing our 401ks and other things We can now start to do on the policy side and this can be truly transformative to states My initiative at the Pew Center on the states is about a year and a half old and for the last you know year and a half We've been going out and talking to states and saying are you interested in joining with us to put this type of a Process in place in your state today We are working with 12 states who are working on this process We provide them the software necessary to do this technical assistance to be able to to implement this approach in their in their jurisdictions Support to help them not only get this operational But also to link the results to policy makers in a way that we think is compelling And hopefully do good things with the results if this is still a preliminary exercise in most states But we think that the early results are pretty encouraging the 12 states that we are are working with we think 10 of them We'll have the initial components of this model operational in time for the 2013 sessions two of them have completed it and have Released reports based on it Iowa and New Mexico and those states have already started moving money from things that this approach showed that We're ineffective to things that really represent better investments for for the citizens And we are confident that as the other states move into the legislative sessions were really the big money appropriations are made That this will continue to gain traction And we're hoping that all of these states will have this success that the Washington state has achieved through this process over The last 15 years this is an approach that was pioneered in Washington state By a research Institute there the Washington state Institute for Public Policy, which is a fairly small research Institute But in the mid 90s the legislature started asking them a question Basically fundamental question We're concerned about crime in Washington state because Washington state like most states was seeing crime as a big problem in the 90s And the legislature really wanted to know what are our options in moving forward in doing this and the institute came Came back in about a year and said Here's what the research shows here's what really are good proven interventions that we think would work in Washington state And the legislature said can you give us very precise estimates of how much money? We'd have to put into each of these programs to achieve the outcomes that that you say we would get and what would really Mean on the ground if we did that so they started developing this cost-benefit analysis model Which is a software component and they've been using this now for about 15 years to really make a series of incremental decisions Moving money away from things that this approach shows are relatively ineffective Two programs that it shows to be much more effective in terms of getting long-term outcomes for citizens Since that time period looking back they're estimating now They're saving about a billion three per biennium which in Washington state is a lot of money in their budget Their crime rate is substantially lower than national average It's fallen faster and deeper than other states the reincarceration rate is lower than other states So they're really achieving the outcome that we want states to have which is they're a safer society And they're spending less money to do it and they're not just locking up everybody and throwing away the key because that can be effective But it's pretty expensive We think this is approach that that we can continue to develop into more and more areas that have that robust research base And the good news again is that there's a lot of areas that have that robust research base The challenge again is getting that information to policy makers Getting it into something that they can trust and giving it to them in a way that can be very useful in the policy and budget processes We are confident that that we're on the right approach to this It will take years to really implement this in in in states There are areas that are our big issues for the nation which the research base simply isn't there yet So you know as as a country moving forward We know there's areas that we're going to have to invest money into to learn what works But in a lot of areas we have that knowledge and the question is just how do we get that information to policy makers and In a compelling way and I think this is a way that we can move beyond all the information We have out there to to knowledge that can be used and I'll leave this off, but at this Thank you all for coming today. I know this is People are nursing one kind of hangover or another either one of joy or one of misery So I appreciate that you came out today My name is Lorelai Kelly I am leading a project here at New America Foundation at the Open Technology Institute called smart Congress and It's not a contradiction in terms It doesn't have to be although maybe it has appeared that it hasn't acted in the best Intelligent fashion over the last number of years All I think the easiest way for me to introduce my project is just to say how it started I was asked two and a half years ago to buy a group of Philanthropists to go up to Capitol Hill where I worked for nine years on global security issues as a shared staff person So I helped both Democrats and Republicans on post Cold War security And it was obvious at that time just that the institution itself is pretty much an antique It really still looks at the world where national security is concerned like World War two just ended like it's 1948 it's it's much more Capable of watching from Napoleon to cross Prussia than it is to deal with complex messy Interdependent kinds of threats that we're dealing with today and One of the things they kept noticing is that their advocacy strategies just weren't working and So they asked me to develop an interview agenda to go up and just talk to staff person after staff person on committees And on personal staffs to say what kind of information do you need what can we do for you? And so I went to interview after interview talking to staff and they are all across the board Extremely frustrated at their situation sometimes feeling demoralized very conscious of the fact that the institution that they work in is has such low regard in the public's eye and It became obvious to me that we were dealing not with just a sort of an institutional obsolescence problem It's built-in filters on the world and lenses on the world are if not old then often looking backward But also it's it's suffering through an information overload and an interesting piece here when you also again look just as Institution what I'm trying to do is develop empathy for the institution of Congress because we do all have it in common Whether we like the left or the right the Tea Party or the progressive Democrats Everybody needs to work within this institution that we have and it's really a fine one that has endured many trials over the decades and centuries, but it Really needs a special kind of help right now and another thing that became very obvious to me on Capitol Hill is that? Despite all these new transparency rules, which are kind of bureaucratic and inside baseball That has busted open even more the amount of information that's incoming and the gatekeepers of the information in Congress as anybody here worked on the Hill are About 25 and the end users are often between 65 and 75 and so they're just they're just not going to use The electronics they're not familiar with it There are strange rules that forbid electronics in some places and not others Every office I talked to you said their members still requires a hard copy in 14-point type on everything I went to an event the other day and this is probably an exaggeration Where somebody said? Hey, we think it's a technological innovation in our office when the member uses the tab key instead of five spaces So this is what this is what we're dealing with and so what I'd like to set first of all is if we can Have empathy for this institution because it is a wonderful institution and it really needs a special kind of intervention and help right now so the this issue of Knowledge and information It was obvious also when talking to staff is that they know that not all information is created equally but the high quality information is not available to them and a way that they can use it and What's also, you know when we move into the political world a social capital, which is something we've been hearing a lot about in Discussions lately with the millennial generation and also just globalization and new media is The people sometimes with the best and the highest quality social capital meaning Interests that are based on common goods and public interests often spend it terribly then they they have great They have great relationships and they have great expertise, but terrible timing So I thought well that might be something that transparency rules can help out with because for example You can be in the room in a hearing now in almost every committee hearing except for the appropriations committees now They're doing real-time webcasting and this again is all changing So what do we do with this transparency? Because the other piece of this and and I think this is really significant important is that I feel like the Transparency movement for lack of a better word this real drive not only the United States, but around the world to bust open sort of the last Cloistered secret areas and government has reached a plateau and that we're finding that transparency without accountability systems built in While the transparency is being created aren't necessarily going to create a better democratic practice I mean in fact it can drive cynicism and then when people know what exists and still the same behavior goes on and on and Then I written interview to friend of mine who's a chief of staff Somebody actually went to graduate school with and I kept saying this question is what kind of information do you need? He said stop stop with the information. He said quit sending me clickable links I need experts in the room and the political incentive to use facts So then you have another set of problems here is that knowledge and especially high-quality knowledge that maybe has second and third order Impacts that is a complex problem doesn't have a very effective political constituency It can't compete in the room with what I like to call mass and money That's noise the noise created by millions and millions of emails and the statistic that We love to look at on this problem is that members of Congress's offices have experienced from 200 to a thousand percent More incoming contacts from the outside world because of email over the last ten years They simply can't handle it. And so what's happened there is the sort of siege mentality Relationships become more important than ever because what they're looking for are familiar filters and high quality Information through people they can trust so in many cases I think what we're gonna have is a re bolstering and a reconfiguration of again Relationship-based influence on Capitol Hill the problem there is that the people who know that very very well are Part of what I'm calling the wicked problem and the paper of Congress's wicked problem is available outside if you'd like to see it is is it's the self-reinforcing problem where it's these very lucrative special interests on K Street and elsewhere who aren't doing anything in nefarious I mean their rights to petition Congress is guaranteed in that Constitution But they're really sometimes the only ones in the room giving you actionable Knowledge and so you go with it because they're looking for a triage system. They're looking for decision rules They're looking for filters and the people sort of with the common good big-picture agenda are either just stuck in this sort of Noisemaking machine and they're not in the room with the expert knowledge when you need them so Something that really clicked in my head and the other my other favorite quote from this set of interviews And I probably did 85 at this point starting to do them in the states and districts as well was Listen, you think about it as an information cartel Information is like a commodity or weapon. It's certainly not a public interest and that should make all of us stand up And sit back because whether or not you think Congress is venal and corrupt I think that there's a better case to be made that Congress is obsolete and incapacitated and That if we focus on the fact that it's obsolete and incapacitated and start to build a more what I'm calling a symmetrical knowledge environment for it one that means something to members because Much of it is located in states and districts then we have another a completely redesigned Reengineered playing field for high quality expert knowledge. Of course knowledge is always going to be debatable It's always going to be controversial But really the very basic the very basics of filtering information through institutional Experience or the wisdom of what's already happened. It's pretty basic and non controversial. I mean just members getting to know If a bill that they're looking at was reintroduced who who are the stakeholders already in the House and Senate? Is that a staff person where where are the knowledge sources there? There's definitely sorting and filtering mechanisms coming into Congress right now, but a lot of them are dealing with the noise I'll give you just some example and they're listed in this paper. They really sort sentiment not substance So we have to start getting out of this whole notion of crowdsourcing and into what's a more helpful notion for governing Which is curation and this sort of curated knowledge is what we need at this point because casting the net wide It is not going to work for us. It's too complicated. It's too much noise And I think figuring out sort of what is this division of labor of participation for policymaking? really is the next step in this open government movement and also the improvement of participatory democracy so the The paper I've written to is a backgrounder on what used to exist in Congress Congress's wicked problem is that The a wicked problem just to define it is is a very complex problem with lots of moving parts But the key is that the people causing it are also the ones that need to fix it so the agents of the problem are also the ones responsible for fixing the problem and in this case It's members of Congress and American citizens So the second phase of this has been looking at the states and this is where I found these Wonderful projects and the gentlemen sitting on the dice is that they're already the states are already coming up with some Really interesting and and hopeful sorting and filtering mechanisms for high quality knowledge and information And that my sense is that because these transparency rules keep in mind just came into effect when mr. Boehner became The speaker is that we're just at the baby steps of figuring out what we can do now with all this transparency If you look enroute around in like the House of administration rules Ethics committee what you see is real inconsistency. You see an institution trying really hard to keep up The rules on communication outward is where internet fits But you've seen some sort of distortions of this it seems to me you're seeing a committee websites being basically used as war rooms a Political war rooms instead of substantive help portals for the institution There are no rules against that and when you go in and you look and you say oh the violations of Use of committee websites is under what's called the franking commission So you go you go to the franking commission, which is who gets to use stamps to send mail who pays for mail leaving offices It's very highly regulated because of the timing, you know before election after election anyway This stuff all makes sense, but it is complicated There's not even a category for the internet on the franking commission website So you have real disparity and unevenness in dealing with technology inside of Congress This at the state level what I'm looking at is is a how do we leverage these? Entities that are actually supposed to be producing knowledge on behalf of the public interest anyway You know the land grant system universities community colleges and I thought it was interesting is why again They have huge social capital because they're based on public goods But they spend it terribly and it's usually just because they have bad timing or they don't know how to get in the room Or they think it's lobbying which it's not Is I did a little bit of back of the envelope research and I looked at of the original tea party caucus And you know the tea party is known for sort of wanting to banish the public square and to cut spending a lot on any kind of public entities Out of the original 60 tea party caucus members almost 80% of them went either to a land grant or a public university Flagship three of them went to community colleges That's a huge piece of Social capital to spend why aren't they becoming somehow the help system the in-state? Surrogate Wrap a response knowledge teams. Why aren't they providing graduate students in accordance with that members committee assignments to help come up with Witness research questions super finding the people. That's the kind of thing we can do now with technology and so You know it's one thing for people to deny You know a lot of evidence on something like climate It's another thing if your alma mater wants to engage you back in the district on sort of global Awareness that that happens very rarely and it happens even less in the district And the other thing I'll close with this is that what I found in talking to member offices is that? Policy staff are migrating into districts. This is something that nobody really keeps track of but I think it points to something really interesting Which is that globalization has caused a need a constituent service need for complex policy thinkers? And I find this out by accident because I do these interviews in the states and I'll get cards from the staff I talked to and and you know a handful of them are Doing policy issues in the district and if anybody who's worked on the hill knows the huge division Between state and district staff and DC staff is this is where the policy staff are and the district But you also have legislative directors flying back consistently and I don't I mean this is my hypothesis But I really feel like globalization has caused a need for real subject matter expertise Substantive knowledge and new kinds of knowledge sharing networks in the states And what we're trying to do is identify those help them become relevant and useful sooner rather than later and Reengineer the expert knowledge system for Capitol Hill so it's far more symmetrical across the board Great, I'm gonna operate from right here. Good afternoon. Everyone. Thanks for joining us and I'm glad to be on this panel And thanks to Laura live for including me in this significant discussion. My name's been frankly I'm recently retired lieutenant general from the US Army last March I work for President Crow out at Arizona State University and I work here in the McCain Institute for international leadership and So by way of background I as a young brigadier I worked in the National Military Command Center where we received a little bit of information from time to time and had to Make judgments on that to the chairman of the Joint Chiefs of Staff the secretary defense and the president and then I commanded our forces in Afghanistan from 2006 to 2007 Operating out of a joint operation center that received information from Around the world from all our inter agencies and from the military to visualize think through sort and and have to make decisions from a Lot of different information in a rapid period Today I work at ASU advising President Crow on leadership initiatives And I work at the College of Public Affairs and in the McCain Institute What we're focused on is international leadership in the area of security Economic opportunity freedom and human dignity at home and abroad and with that we're working on a project Building off of work that's done in Arizona State called a decision theater The decision theater was applied locally to do some work on helping leaders visualize Water policy and what to do with water in a constrained environment in Arizona and also to do urban planning with the placement of Different buildings and different designs in the Tempe and Phoenix area to be better informed about the placement and the growth of the urban environment What we're focusing now is a decision theater here in Washington DC as a part sponsored by ASU But as a part of the McCain Institute to help policy leaders and help the Congress and help others We know that people make decisions but better informed people make better informed decisions as Gary and Lorelai both pointed out And so what we're thinking about is the richness of data that comes through business models through social media through YouTube through various means how that data is turned into information How information is turned into knowledge how knowledge is turned into understanding and then from that understanding decisions are better informed and and we look at Evidence-based which is the facts that we'd like to get at hand and push out the special interest or the emotion Tied to it or perhaps the money behind it to push an agenda and really get to the evidence-based facts But then tie that also to expertise and the expertise coming from experience and judgment It's taken the 25 year old staffer on the hill that's got a lot of facts at hand and Giving it to the member who's got a lot of experience and judgment in the policy arena and combining the two to get the decisions So in the decision theater what we're Focused on is is how do we visualize this knowledge first? How do we put it up for people to see it really understand it and then what is the impact of the knowledge that you're working with? How can you look at cause and effect sort of if then work as you manipulate the data interact with the data and the knowledge to To push it around and then what trades are associated and what are the risks associated with those trades? if I put more money into Secondary education is that going to help more kids go to college or should I start with even early learning and start with young people? Or should I try to distribute scarce resources and education across all grades and how well is that going to work sort of what? Gary's done with crime and and in Washington so evidence-based applied to experience How do you then do the trades and the risk analysis and then see and understand the complex? Relationships that are out there more and more we know that in this complex world Where and there's a lot of networks engaged with it and single solutions don't work But networks applied against networks and can help understand the complexity and help solve complex problems So we hope to create networks that can feed the decision theater and and have a richer content That leads to understanding visualization and then move to decisions and once you make that decision you go right back into a cycle where you Assess that the impact of that decision and what's the second and third order effect and where do you move forward from there? So in this complex world, we hope to assist through this decision technology Thinking and approaches both at the McCain Institute and at Arizona State. We look forward to discussion with you about any of that Thank you Great. Thank you I want to thank all three of you actually for for bringing up some great ideas and for provoking what I hope will be a lively discussion So I want to turn to questions in just a minute and comments, but but but maybe first pose one myself You know in listening to each of you I was trying to sort out what Kind of both what the problem is and what and what we're trying to solve is and and I hear different things and I and from each of You and but things that connect across each of you or each of your presentations and would be interested in how you react to each Other in a sense and and in particular I'm I'm I'm trying to understand whether we should think of this as a demand side or a supply side challenge or both Or a structures Problem which I think has been what you're what you're talking about that you know that there are some supply side problem I mean, I think that the answer in some sense is going to be a both and right but but but as you sort that out or kind of clarify it You know I hear what you're saying Gary it sounds almost too good to be true You know that well if we could just produce compile all this research and produce it all that there's there's not an appetite for it But what's interesting as you described it I thought and not knowing a lot about it is that in Washington State there was a demand for it That it was and they actually created it and now as you try to take it To other states, it's pew that's going to try to try to bring it And will there be that that seemed demand and what will it take to sort of generate that demand? We might take a first demand exists pretty much everywhere I think the policymakers are almost uniformly frustrated by not having the knowledge that they want to have to make informed decisions I mean virtually all policymakers will want to do the right thing so they bring their ideologies and experiences to it, but but I Think that the frustration that they have is what? What really is going to help me make a better decision for the citizens that I represent? So I think that the demand is out there. I mean we have had Some interactions with almost every state in the last year and a half saying hey, we've heard about this Can you tell me more about it? but we also want to make sure that those linkages exist that that Lord Lee talks about because It's not just a simple factor of if we if we build this it will come you have to end up Setting up an institutional structure in a state that that deals with some of these these challenges Bringing this information to people getting them through the gatekeepers getting the gatekeepers confidence in this and also making it visually compelling So that people can understand this information when when it comes to you So bringing this information to people is only part of the battle and you have to view this holistically Because it has to be something that that really provides good information But also provides it in a way that is useful to the institutions that it is coming to and it's transparent enough that people Understand it, you know and won't view it as something that is partisan because we're trying to keep this very non-partisan And very objective recognizing that the dent policy makers will bring everything else to the table which other ideologies their district interests You know that their own experiences And what we're trying to do really is sort of I think raise the batting average of evidence If you look out there now batting average of evidence is probably you know, oh 50 on a good day We'd be happy to get up to maybe 300 I mean that means that a lot of times other factors will trump data and knowledge but if you bat 300 long enough to get in the Hall of Fame and We think that that is probably a good initial goal for doing this. Do we need to have a switch out real fast? Okay? I'll just answer that is is that it seems to me there's a couple of problems here is is that yes Expertise who's informing the influencers is always going to be controversial and hard-fought Like when I worked in Congress, I was always amazed at the people with the best information always sort of sent me their dissertation six months after You know the legislation passed through Congress. It was a timing problem It wasn't a trusted relationship problem, and it wasn't a substance problem So the transparency piece might really help solve that timing problem for a high-quality knowledge That's based also on relationships the locational aspect of this and when I say that I mean this disaggregated system I don't know what it's going to look like and it's going to look different in every state because it has to have be you know grown locally I Think we'll create an accountability system for High-quality ex robust knowledge that doesn't exist right now when I went down to Tempe to look at Decision Theatre Was loving it basically what it is is seven simultaneous screens Five of which have tabular data on things that impact water scarcity in Arizona It could be the Colorado River flows, you know the needs of Las Vegas Utilities and you and the evidence is utility bills. I mean, I don't know how much you can argue with some of this stuff climate patterns And then what was lovely and this is again real-time trade-offs rarely happens for members of Congress and and Congress and politicians suffer from what? social psychologists called loss aversion which is the willingness to To to sell off or abandon future gains to prevent immediate loss This is very common I mean if we can do anything to bridge that gap and help people make the hard trade-offs up front What you could see on the final screen were pictures of the trade-offs and it was you know swimming pools and golf courses and bathtubs and lawns downtown and you know just people living in a community And so if you decide to do that, you know, you could tweak any number and it's on a matrix So it switches everything and you could see then are we really going to trade-off, you know Showers for swimming pools. Are we really going to trade-off if if the Colorado River goes down this low that kind of real-time? Accountability system. I think is the only way that high-quality knowledge will get some political chops And again, it's about timing and it's about being in the room in a way That's useful the other thing is just the capacity of visual data to tell a story in a matter of seconds It's huge and ask any, you know under 30 person About the power of visual data and we need to figure out ways to translate You know numbers into pictures and keep in mind and this is just a story. I love from reading this book Something called like Leonardo's laptop not remembering the name of it But it talked about how when the printing press was Developed, you know, this is Gutenberg Hundreds of years ago. What happened was text become became far cheaper than pictures to print and so we went off on this into this, you know Centuries of using text instead of accompanying it with the picture because prior to that it was Galileo's Astronomical discourses. There was always a three-dimensional picture and a story underneath it We have the chance as humans who crave visual stories To now put sort of both halves of our brain back together as I like to think of it You have the right brain and the left you have the linear and the visual that to me I think we have no idea what that might do for us is to tell the story in a far more compelling robust way up front politicians hardly ever get that opportunity and you know it creates an accountability mechanism So so your react both of your answers sort of raised a question. I want to put to Ben. I guess which is You know one on the one hand it seems as if we do have a knowledge problem and an expert expertise problem and information problem but but then we also have in a lot of ways a people problem and And that people the people on the knowledge creating side the suppliers Haven't adapted to changes in the way our society works in the way our institutions work And those on the demand side aren't capable Therefore a figure and how to interact with them in an effective way the decision theater is intended to try to In part at least to bridge that it seems like as a but I'm wondering how you think about What additional people we need to bring to the table or how do we it you know And this goes maybe in part to Lorelai's point of sort Well, if we could only leverage the resources of the land-grade institutions and as a former college professor I think about my colleagues and I think It's not going to happen, right? You know even if you get them to do something more it isn't going to be what you're talking about It isn't going to be and you're trying to do this at ASU. It seems to me And so I just wondered if you could speak to that a bit well Firstly to go to your first question that you asked Lorelai and Gary about what's the problem? The question really is are our policymakers Making the best decisions they can and we see in business that many people who have organized information Have applied metrics have a system in place make very good decisions and businesses flourish when they have that kind of information with what we're trying to do is in this Data-rich environment that comes from everywhere. It's not just one printing press producing one document But it's many talking to many simultaneously is how do you harness this this knowledge? How do you get the universities and other areas of expertise? To some of the work that you've done about the old think tanks that were really just think that really great Folks who did great analysis. How do you reach into those that are doing great analysis and bring the visit cut through the Clutter because we're filled in this world today with all the clutter and the noise How do you cut through it and harness that and and put this in context so data to information information to knowledge? Now in the academic setting you've got to find folks that are passionate about the same passions and you've got it also a help You know some people say well How does the military rapidly organize so much information to a rapid output part of that is standardization? But you have to be a little careful in that over standardization just crushes innovation But innovation without any standardization is sort of energy So you kind of want a sweet spot it seems to me in between you want to get these great brains across our country engaged in the Dialogue doing that someone else can possibly do the fact work when I talked about the adding the experiential layer the Expertise level you you want folks who get this Depth and this huge ball of knowledge who look at it as Gary's guys are doing a pew and cutting through and say this is what really works This is what doesn't work and here's the trade So I think we have to look out across America at all places where knowledge exists where analysts exists and harness that power And that's what we're trying to do through decision theater Let me start opening it up for questions and comments if yeah, and I may we do have a Also just want to recognize the under who's at the Sunlight Foundation who agreed to come down here and help us today Sunlight Which is another wonderful resource in this realm Hi, this question is for Laura lie I'm Loretta Goodwin from the American Youth Policy Forum And I wondered if you could talk a little bit more about the timing issue and you talked about the dissertation arriving you know six months down the road and I was wondering about the need for knowledge in Time and where the onus is on both reaching out for that knowledge and also the knowledge coming in so I wondered if you could talk a Little bit about that sure one of the the One of the things I realized actually lately I went up and talked to the ethics staff is I didn't realize that there are actual certain restrictions on what members of Congress can ask for from their constituents because They're on the side of caution a lot not to create conflicts of interest and what ends up happening is this chill effect So nobody talks to anybody because they're afraid they're going to do something wrong Well, the I think this is has hurt this knowledge sharing quite a bit Because it just makes it more difficult my sense is that There's some simple steps forward that for example to create more symmetry of access to high quality vetted information For example between committees and personal staffs on the Hill committees are just they're the well-fed The the people who get all the trips who get all the visits from the people coming all the Aspen You know steady groups and hardly anybody does this kind of same care and feeding for personal staff Yet the personal staffs where the member who's going to write the speeches for the member who's very often going to do that What I you know, I guess you call shotgun work, which happens at the last minute There also needs to be more symmetry between policy staff and the communication staff on the Hill So it's creating like we experimented with this a couple years ago with something called the Afghanistan congressional communications hub Which was simply trying to even out the knowledge available about civilian engagement in Afghanistan because you had the military repeatedly Testifying that there was no military solution and that they needed some kind of way to build social resilience in Afghanistan But Congress kept basically throwing troops at the problem I mean they did fund a lot of of non-military stuff But the problem there was that it's pretty soon you had the armed services committee writing the civilian benchmark language That just that shouldn't happen. It's a it's a problem of obsolescence in the institution that it doesn't share Knowledge that's a general concern. Why can't a university out there across the country that has a bunch of returned National Guard and a couple retired officers plus a lot of people Maybe who have familiarity with Afghanistan build some kind of a knowledge hub that bridges that gap between committees and personal staffs This is what one staff person said We we need an eBay of expert knowledge for Congress Not like another Sotheby's or another like high-end retail auction house We need something that's matchable by location where it matters when it matters that the person on the user end gets to Control I guess it doesn't seem to me to be too much to ask of the academic community in this country And I came out of an academic setting as well To please step up and help these institutions evolve We need it so badly and there's a unique and indispensable role for people who are looked at as experts To help help on this front right now. What about dedicating a team of grad students? What about dedicating what we're calling knowledge fellows to district offices where some of their job is simply Inventoring the expertise available in the members district This is the kind of way that you cut through the noise They're already doing there right now. They're sorting was with the area codes and zip codes That's still it's not very useful when you have a real precise question about girls education in Afghanistan, right? There are ways we can do this. This is a hard problem. It is not an impossible problem And for me to I grew up it or I was born in San Jose. I came here from Silicon Valley I want Silicon Valley to step up and start, you know Like letting go of this sort of libertarian dream world that's furnished by apps for everything That's not going to happen with our governing institutions It's not going to work. There is no app for Congress. There is no app for the executive branch We need to figure out this problem and it's a perfect one for people who think in systems Which is 70% of the people who live out there It seems to me that dealing with academic institutions is really an interesting issue because I mean on one hand Academic institutions have a huge amount of next expert knowledge. On the other hand, they're incredibly siloed all of the academic It incentives for faculty are to hoard institutional knowledge because what you're based on for your tenure decisions is how much you're going to publish Teaching is Factor into that, you know sharing your your experience with Congress not at all If people would probably advise you against it. So I really think that there needs to be a couple of institutions Arizona State and others who really Find a way to do this and are the innovators in doing this because if someone has to do this I mean, we're not going to have more expert knowledge in the future or more expert resources than we have now I mean, it's there just isn't going to be more money for this stuff So we have to deploy the information we have much more effectively and that's going to require you know looking at the incentive structures We have you know building the network solutions that we need to have because you know Congress and all political institutions are very highly backward and they're really having a way of bringing this information together in ways That are useful and usable We've got a ways to go on that. I just make a quick comment that President Crow has focused the university on what he calls inspired used research So if the research isn't really going to be usable, then they're not really underwriting the research that's ongoing and just on the timing piece I think it's so important that as fast as cycles of information are flowing now That leaders have to really stipulate when they need information by and that takes some of their work to envision What's up in front of them? What policies they need to make by when and help and inform those who are making policy or Contributing to policy and when decisions are needed by because this Lorelai says you know to have information six months later Thank you, but it's going to be a great historical record, but it didn't inform the policy when it was needed So leaders really have to stipulate when it's needed or rules have to change to to help move information faster And Gary just if you could quickly along the lines of the conversation We're having can you say a little more about the mechanics of how your Project works in terms of who who brings the information forward who makes sure that the information is available at a time that's Useful for the decision-makers in terms of pulling the information together right now We're working with the Institute in Washington State who's developed this approach and they're combing through the published research But both you know the journey are journal articles as well as information that's available on the website But but you know to us the exciting thing is that we know that the states have a lot of research That's never been published some of it is very good research So as we're getting into more states then we think we'll be able to share that information more, but you know in In our work, you know that that's only part of the solution the other solution really is to you know Take that package information and then build a structure within a state That is very responsive to policymakers can provide this information in a way that is useful to them But within the time periods that they needed on it because you can't tell a policymaker Yeah, give me six months for that decision. I need I know you need to make tomorrow and I'll get you something You even if it's not perfect information. It's still much better than what they're basing on now, which is you know information facts distraction distortions and everything else that's out there, so Or turning to the intern sitting next to you and saying what do you know about the nonproliferation tree? Exactly what happened in Hill offices Sorry scary it and we'll turn it back to another question But Ben just quickly with respect to the decision theater the same question I guess how you're going to kind of deploy it or you know optimally I guess you'd want to sit policymakers to sort of say look We need to figure out a place to go to sort these issues out and that they could come and convene a session there But is that right or we're going to begin with an excursion with the Congo and and look at the issues associated with Congo and The way in which we're getting the information is we're actually having a class at ASU for credit Constructed a policy design class where the students are going to learn about developing policy How do you formulate policy, but they're going to collect information on the Congo in a lot of different areas and Proliferate our databases with the information they can find sort of what the Washington Institute is that what you guys is doing for for the legal system and so We will look to see how that operates, but we'll also look to see where other Places of expert knowledge reside about the Congo non-governmental organizations Different you know doctors without borders or whomever who's in there that has information that they want to share about what they're observing that's happening in the Congo and so we want to have a layer of information say it's a Geospatial type information that's reusable But as others have problems they want to deal with they can come and layer in their information on top of that And we can make it cost-effective for policy leaders to use it over and over again if each excursion is prohibitively expensive It'll be a short-term decision theater because it'll as Gary says our resources are coming down So we have to figure out ways to do things more economically and move information easier for people to use But we're going to do it through using the students and the faculty at ASU to populate what we know about the Congo and expand on that Lorela, I really liked your report that I was in before the 1995 change of rules and saw a Congress at a time that saw the value of shared information of Having information available with without trying to control it Since that time we've seen much more having a almost of Well, I'll say a military approach to saying we want to get here What do we need to do to get this policy implemented and if it means ignoring certain information that gets ignored You point out that a lot of that is done through rules of what can be funded What organizations can have staff who they can talk to in effect? I think all of you are talking about waging a guerrilla war against this existing structure Is that feasible or do the people making that's the policy decisions who well understand that you might be a threat to Particular policy objectives Can you can you win that war? Well from my perspective? leading 26 nations called NATO in Afghanistan is really hard to get consensus and then when you ask the pakistanis to come to the party and you bring in the different elements of our own government USAID Department of Agriculture and and state and others Getting consensus is really hard But when we operated in the joint operations center and laid out visually the trades and what we're trying to do and where we're Trying to go we got a motion out of it. We got agendas either NATO personal country agendas and caveats We got that out of the way. We got consensus around the table and moving forward based on facts Not based on any other any other things So at least if you can start with facts if you can start with good information The Geary keeps bringing us back to at least you you have a fairly firm footing and hope to move forward in a Consensus manner short of that. You're just going to be fighting over either agendas or Evidence that's produced By the lobby group that you paid for to have to do the work or however it gets to you so I think if you start with facts and you start with visualization and you bring every they have a Transparency where everyone's brought to the table where Information is not a commodity, but information is shared and it's available to all I think that starts moving you in the right direction where we still be seeing the threat probably But that's not our intent. Our intent is to have a safe place To have informed decision-making take place based on evidence and experience Understand where you're coming from but I don't think that what we're trying to do is sort of a gorilla war against the institutions I think it's equipping the institutions to do what they are designed to do but to do it in a fundamentally better way by providing them with answers to questions that they have to deal with and because Every legislature has appropriation staff whose job it is to try to inform members decisions about well What should we do in the criminal justice budget? What should we do in the early education budget? But you know those staff don't have a lot of you know ability to go out there and research You know different alternatives that they could bring before members So if we can bring that together and build that into the process that already exists there strengthening into times But also just providing them with better tools to do the job that that you know needs to be done and is being done Then I think we can get by without you know Exasperating the political conflicts that are there I mean those conflicts will continue to exist members will make decisions for other reasons than this data And there may be some folks who are threatened because they will view it that well This information shows that my program stinks, but you know, I would still like to get funding for so you know You will still have battles over information, but I think that if you do it in a way that is transparent and is based on very sound principles then it becomes hard to refute I Would you say that I another hypothesis that I think was at least partly borne out last night is I think we've reached in our political system, but especially in our institutions and especially in Congress that we've reached the point of diminished returns where money and noise are concerned That the point of diminished returns is a great economics term actually it took me 20 years after I took economics 101 to actually understand what it meant in politics is that it's the point where one more of anything is counterproductive It's not getting you anything extra, and I know it's oversimplification to say that all this anonymous money from the super PACs You know was worthless. I know that's not true because there's a lot more complexity. There's down ballot races There's a lot more out there than the president's race But if it makes the point that listen there is a lot more subtlety and a lot more leverage Influence leverage that we've hardly accounted for and one of them is again How do you spend high quality social good? Relationships the the other piece of this that I think is really important so we see like money's reached its its maximum saturation point perhaps is that Social media is in the sort of dream world I'm sounding like a cranky hill staffer here if you think like Coney 2012 is gonna get you helicopters in the Sudan for peacekeeping missions Or if you think simply passing an act to feed the maw of the anti or the genocide intervention movement Or whatever movement is out there these people cannot hear this the sopa peepa fight last January I think is a good object lesson. How many times are you really going to shut down the internet to get what you want? Are you going to maybe make sure sometime that there are technologists in the room when the legislation is being written? So it's again It's it's sort of the it's the supply chain management problem of knowledge within the process of policy This is a much harder problem, but you know if you're looking at legislation sopa peepa went down hard It looks really really pretty seamy that just one narrow slice of interest was very represented in the legislation But within a couple months a bill passed with a national security label slapped on it I think it was called SISPA which actually eliminated freedom of information Act requests and I think in other ways was more egregious and very damaging So again, it's this problem of having standing support accountability systems for using expert knowledge And and we have to figure this out like around the world We're not a unique case citizens expectations for meaningful participation in their shared future has far outpaced government's ability to provide for that far outpaced you have Iceland recently which sort of curated a new constitution with a lot of participation you have Dartmouth College doing the sort of Comparative analysis for the Vermont and New Hampshire legislatures. You have Washington State the institution He was referring to I think was created by the legislature itself, wasn't it and and the other thing was about the office of technology assessment It was so interesting is because they didn't work on a problem unless a Democrat and a Republican Shepherded it through that's not true of the other support agencies today So you can build in empathy and bipartisanship and how knowledge provision is structured to share And that's what we're not doing very well. So this stuff isn't rocket science, but it is complicated and we've done it already Lore like this question you sort of begin to address in your paper Just about every state and the US Congress have access to Nonpartisan legislative research organization That was set up just for this purpose to have analysts at hand who are policy experts and who know about the timing as well and Obviously, they're not working out there. They're in Congress. They rarely seem to be the primary source of information for members offices or committees Why would any of these new efforts be different or what do we need to do differently to make? The dream of this sort of nonpartisan effective policy Body really be used as it's intended to be used You should you should ask because you've had such incredible experience in person You're totally right that almost every state has a legislative research office that it's created to do research and to do evaluations and do policy analysis There's several challenges with them one is it's a normative institutional challenge When these offices were set up when that field was set up That this was bad really most of those offices were created back in the 60s and 70s And we were still really in kind of a positivist perspective as those norms of those offices were being set up and so a lot of them have a very strong institutional norm of independence and on one hand that is Lottery because They want to provide objective information on the other hand if they're not networked into their own legislation Then their information isn't going to go any place it and that's a huge problem for a lot of those offices So, you know the question as we're going into states is to recognize what resources are there But to put you know the tools that we're developing into the process Frequently in a different place that some of those offices are are working with us because they work well with the process for example in New Mexico the legislative Research and evaluation unit is part of the appropriations committee Which basically guarantees that their work is going to be highly focused on things that matter because that's what they're doing in other offices in other states those offices are sort of off to the side and You know that they do great work some of it has impact, but it's not really where we view You know this work should be for for what we're trying to do You know I In an ideal world you'd go back and retool some of those those arrangements That's not something I want to spend my time doing with this because that's a bigger and long-term challenge I think we have to be mindful of the challenges that that are out there I think that there are some great examples where states have come up with institutional arrangements that are very functional And what we're trying to do is replicate those successes and and move forward on that And if we can drag the rest of the field along behind us all the better Because you maybe say a word about a place a state that's doing as well or where the institutional arrangements work because this is a concern I've had looking around the country and looking at Washington as well, and I now having left academia. I run a small Truman Scholarship Foundation which supports young people committed to public service and it's been interesting to watch How those who are committed to public service perceive what public good public service might be and where it is They're interested in going and working and they're not interested in going and work working in the kinds of agencies that provide expertise and information They don't it doesn't seem to be highly valued. It doesn't seem to be something that provides kind of useful career trajectories for them They're not sure where it's going to lead and since nobody's using that work anyway Why would they go kind of bang their head against the wall and and so as a consequence? I don't know if that is replicated across the country But there's a worry of are we getting good good people in these agencies? Can we attract them or are they kind of in Loreley's? Loreley's language kind of antiquated If you talk to people coming out of school, it's the one thing that I think is universal is they want to make a difference So so I think that the hunger among people coming out of schools really good schools really good Students coming out of them is they want to work someplace that that makes a difference So I think our challenge in designing institutions is to make sure that our Institutions are actually doing that that they are making a difference and you know I think as Loreley has found I mean many of our institutions of governance are hugely dysfunctional that they were based on you know You know an institutional structure that made sense in the 1700s or the 1800s or the 1940s or the 1960s But you know we're living in a different age now knowledge is everywhere The problem that expert you know use of information is huge I think it's up to us and anyone running an institution to come up with a way that that our institutions function much better And in terms of places that that we're working at that. I think work well I mean we're currently we're working in 12 states and of those states I put them in sort of two boxes one is that we're working sort of With the executive branch to to get this operational and the advantage is that they have the data and that they know their data And it's technically easier to implement the model because they know the data the challenge is getting those those offices to network with the Legislature in a functional way in the other half the states were working much more with The policy side of the world and in Mexico is a great example of that I mean we're working directly with the Appropriations Committee and the research staff who are assigned with that committee and They're doing great work. I mean it's in some of the states we're working on It's taken those states a year to get the model initially up and running New Mexico did it in about four months And they've already issued two reports based on it And they're already making you know changes based on it because they understand that the questions that their members have and They're tying this work directly to the process So our lessons learned is it's really important where you put something like this to make sure that those network linkages Are there from beginning because it's incredibly hard to create them if they're not there So, you know, I think those offices are there. I think they they do good work I think that they face some challenges and it's a resource that is there It's a resource that needs to be used more than it is. There's some challenges, but I am optimistic A lot of people don't realize that who initiates information or knowledge sharing In Congress is a huge issue. I've often noticed I've worked with the military for so long Is the military is not going to ever initiate a huge public education campaign on behalf of Congress yet They have some probably the best Experiential memory of the last 20 years in foreign policy And so we have to figure out ways that people who can initiate do and this is you know citizens Who live in your district who are experts like we need to figure out how that how to help them self-organize Nothing is going to sort of make it in politics in the end as you know unless it deals with turf and ego And how do you make knowledge? Fit into that or make it more alluring and make institutions more sexy one of the things I found as the kind of resident Old person in the open technology Or I work I'm about 15 years older than most of the staff is that this realm that needs to come together and figure Out this problem is populated which just to pick a good metaphor is sort of the the Jedi's on the Yoda's and not the Jedi Knights And the Jedi Knights are the people who might be retiring another five to ten years who are immersed in these institutions and their height of dysfunction and That we need to figure out how to get them in the room So they get some empathy for each other that like I would like an app But maybe I just need a basic human resources sorting app not a fabulous, you know Inversion boots mechanism that also it just I'm I'm making fun of my colleagues But wonderful the wonderful potential if the if the these things meet somehow and a more balanced way All this information all the social media is not going to make a huge Difference in the end unless there's an organization behind it. I think that's one of the things we're finding is that it's still going to take the same old Political organization based on social networks and who you're talking to and accountability and the other thing I'd point out is That members are home in their districts, I guess 10 to 13 weeks more now than they were two years ago That's a huge huge opportunity to do new things with them And a very few offices that I've talked to say that any kind of knowledge Opportunities are presented to them something that's useful for them Somebody just coming in and saying this is what we have available to you There's only two offices. I think out of 80 interviews that said their university system and their experts are really set up To help them when they need it That's that surprised me. I thought that would be a far far more healthy Mutual relationship with time for a couple more questions if they exist Maybe I'll pose one then to kind of to kind of close with it and and it builds on what you were just saying Laura Could you say a bit more about who these knowledge brokers should be or the kind of the folks who are going to? curate knowledge Where we should look for them and then and a bit more about how how we should see a use for technology in Making this all possible. I Think that a knowledge brokers in the institutional sense I think there's a way to help think tanks This is the you know the the center of the universe where knowledge ecosystem is Washington DC, right? We have a an infrastructure here of think tanks a boutique issues big issues anything you can imagine that doesn't exist anywhere else in the World I think we need to evolve a system where the think tanks are much more nodes they're the proximate nodes for a system that's out around the country because Being proximate to Congress is always going to have benefits right we can run up there and third you know in 20 minutes and Somehow coordinate that so that the members have both sort of a set of trusted vetted relationships here By topic by issue and you also have the the turf issue taking care, which is this is also connected somehow to my Consituents this is somehow connected back one of the things that came up over and over again and I'll write this up in another paper in the next few months is Is The what these staff are asking for is really pretty simple and some of it is very technologically available with things like simple wikis right now members not only need to Deal with traditional press broadcast media though the one to many but they're needing to deal with like Micromedia hyper local news something people asked for repeatedly with simple press lists of bloggers and tweeters in my district and Sorting their blogs and posts and tweets into a press clippings for micromedia now This is the kind of thing that might make your head explode, but it's it's very doable And what I would argue and this could be contributed to the state level there could be different levels of access There could be a public-facing But I do also think there needs to be again not all information is created equally and even Wikipedia right now is having to deal with Their editing process being gamed by people who can afford to hire people to go in and change entries all the time This is going to be an ongoing problem is that if you can game the noise or game the Participation with money my sense and I'm going to just be a complete purist on this is there can be no Money involved we have to get the money out. There can't be a compromise on that This has to be people maybe a group of foreign service officers who set themselves up to be your sort of rapid response situational awareness team It has to be transparent who's making money off of something or not. I mean just that alone I think would be really helpful. Well, the stakes are too high For it to be gamed like that on a lot of these issues I mean the fact that like, you know the start treaty two years ago It really shocked me was held hostage shoe tax cuts and unemployment benefits. I thought was the Like how could we get to this place where nothing is transcendent anymore? And I think you just we have to take some hard stands where you know, we're in a world of social capital now We're going to spend relationships Not money money has reached its point of diminished return. It is hurting us badly We need to get it out and figure out how to start again And I have a friend that I taught peace studies with at Stanford who said he's a Pacifist not because he doesn't want to you know shoot people's heads off, but because it makes him be creative Same with vegans that I know it forces you to figure out different things And we have to figure out a way where politicians see their successful future embedded in the destiny of the people They disagree with that's the only way this is going to work and people just need to start standing up for the public interest Why is it that the only federal institution that really feels comfortable talking about teamwork is in uniform? That's just not a healthy place to be as a democracy These kinds of things Adapting the lessons of the military into the civilian world is another huge challenge for us because they have so much experience on creating resilient societies All of that anyway not to to go on but there's there's a huge amount of political capital to be built here I think if we think creatively Great. It's a good noob to end on. I want to thank Lorelai first off for an excellent paper and for organizing this And Gary and Ben for being part of it and thank you all for being here. Have a good afternoon