 Thank you all for coming out today. I'm Lenny Mendoza. I'm a board member here at New America and also a senior partner at McKinsey and Just excited to help kick off this discussion today That's pretty timely and important about the value and value creation and the usage of open data And it's it's great to see so many people here today Especially right before the holidays It's I think a combination of the the reflection of the interest and opportunity in this area as well as the challenge Of how do you actually make this real in the role of government in helping ensure that there is value created? We have a great set of speakers today that reflect the heart of the issue from different sectors So we've got folks from the government the private sector and the media and NGOs here to talk a little bit about How do we ensure that there is value in open data? And we're going to start with Nick Sinai who I'll introduce in a moment of the White House office of science and technology policy We'll hear from my colleagues Michael Chewie and Deanna Farrell And then we're going to turn it over to Alex Howard of O'Reilly media to introduce the panel engage a little bit of a Discussion with you all to close out the morning I'm not going to do a big long windup because we've got a lot of great speakers that are going to give you a Sense of what we're talking about But I want to do just a couple of thoughts about what you're going to be hearing from them in terms of the key points that We want you to take away As we all know including in the headlines today open data is often talked about in terms of privacy and security But it goes much beyond that and the way we want to talk about today It's not in those issues although we obviously touch on them But we want to talk about how open data creates value through improved decision-making new products and services and Accountability a government in this is obviously extraordinarily important stakeholder Plays a vital role as a catalyst a user and a regulator We'll talk through some examples of that But we want to talk about the role of government in that ecosystem to ensure that there is value That's created from this power of open data So let me introduce our Keynote speaker who's going to give us a little bit of a sense of what to think about this It's my honor to introduce a Nick Sinai who's the deputy chief technology officer at the White House Office of Science and Technology policy He helps lead the president's open data initiatives to liberate data to fuel innovation and growth and helps The federal government be more transparent Participate participatory that's a big word and collaborative before that Nick was in the FCC and before that He was also in the private sector before joining the administration as a venture capitalist And as an executive and advisor to startup technology companies So he's got a nice range of experience here as well as a real important role in the government to talk to you today So my pleasure to introduce Nick please join us Great. Good morning. Thank you for that introduction. It's good to be here today as many of you know One of the president's first priorities After taking office was to build a government For the 21st century that has traditionally been in the 20th century and he believes that this work will help us be more effective efficient Data-driven and transparent and so the topic of today's Conversation about the value of open data is a excellent example of how we are delivering on this Commitment and as a valuable national asset government data should be open and available To the public to entrepreneurs to scientists Instead of being trapped in government systems Taxpayers paid for this information they paid for these vast troves of government data and wherever possible should be available to everyone and at a basic level the federal government collects or creates a whole range of data from statistical economic financial regulatory Programmatic research development a whole a whole wide range of data But many of it's trapped and even when it's technically available. It's kind of hard to find Hard to understand and and hard to reuse And so over the past few years the administration has launched a series of open data initiatives In health energy education public safety global development finance To to open up this data in these areas that for the that for the very first time We're very hard to access and so recognizing this the president signed an executive order in May This year making open and machine readable the new default for government information Directing historic steps to open up data and so under the terms of this executive order and and related open data policy Newly generated government data is required to be open and machine readable as the default While continuing to protect privacy and security And so for all of you who aren't data enthusiasts like us I imagine most people are in this room. Otherwise you wouldn't be here, but for those who aren't let me tell a story That I think will will help illustrate why I'm so excited and why I hope all of you should be so excited as well so this is a story about Lewis and Lewis is a farmer in in Indiana Columbus, Indiana He has over 5,000 acres of farmland and he grows blueberries and There's about a five-day window for Lewis to plant his crop sometimes depending on the weather He gets rained out. Sometimes he plants and the seeds gets too hot and the seeds die So a string of bad weather can mean the difference between a successful crop and and and severe financial losses and On the flip side of Lewis is a gentleman named David Friedberg so in 2006 David was a Google engineer and he was driving on the Embarcadero in San Francisco and he saw a bicycle rental hut and It was raining and he was thinking, you know, how does a small business that rents bicycles deal with? Something they can't control the weather and so he was inspired to start a company that uses better data and better Algorithms to help American businesses with what they can't control the weather sorry about that So David started this company to offer weather insurance to businesses And so this company climate corporation offers weather insurance that pays farmers when they don't get enough rain And pays out quickly enough so farmers can afford to replant a different crop and climate corporation allows farmers like Lewis to plant higher-risk crops like blueberries that may not be a good fit for federal crop insurance and Climate Corp also offers apps and tools to help farmers kind of plan and manage Their crops and the company has over 200 employees today They have offices in California and and also across the Midwest And this company wouldn't be possible if it wasn't for government data climate corporation uses Historical weather data forecasts from the National Weather Service Terrain maps and soil types from the US Geological Survey crop yields from USDA weather and earth observation from NASA data from from across the US federal government and I love hearing stories like this of David and Lewis. It's just a great example of the power of open data and what we're going to hear more about today And it this isn't just about farmers I mean this is happening across different sectors of the economy and whether it's its small businesses or large businesses As we'll hear more about today. This is really a broad a broad theme. I Think David's and Lewis's story is great because it illustrates three things First is the federal government We need to make it easier for entrepreneurs like David We should make it easy to find and use any government data set and we need to make it easy for for for entrepreneurs to find data that Is technically available, but isn't isn't yet online, right? So that data that could be made public and so that's something that we are committed to do Secondly open data is not a It's not it ends unto itself data doesn't do anything, right? It doesn't teach your kids It doesn't improve your health and and and data doesn't help farmers, right? Farmers have had a lot of access to data previously So data is only useful if you apply it if you apply it for a customer benefit or or a public benefit And so what's truly exciting is is not not just what we're doing inside of government But how the private sector is taking our data and creating all kinds of innovative things and so whether it's companies start-ups citizens Financial products software new comparison engines to help consumers make better decisions New stories and journalists breakthrough science scientific discoveries There's a whole piece about about greater scientific discoveries that have actually significant Economic impact That's what's really exciting is this is this vast cornucopia of innovation across diverse parts of the economy And I think the the third thing that's exciting about David's story is is that we can't always expect anticipate all of the uses of Federal open data, and so I think that's that's it's not like Noah for example was thinking about Farmers and weather insurance and apps for for farmers when they're when they're thinking about their earth observation systems And so this this unanticipated innovation Is such an important lesson from the story and climate corporations just one example, right? So if you take companies like Trulia and Redfin and Zillow These are these are companies helping consumers make important real estate decisions And these companies use data from the Bureau of Labor Statistics from the Federal Housing Finance Agency Census Bureau But they also mix it up with local data, right? So local crime data zoning data tax data, and I think that's what's really exciting is that the mixture of Of data and then the algorithms and the and the value kind of created on top of it So I'm really excited to see that today a research institution at NYU called gov lab Is announcing the beta launch of a project a list of companies called the open data 500 So they're putting together this list of 500 companies that are powered by open government data and and So this is kind of a broad list of companies. I obviously won't read all 500 But here I'll just give you two examples from that from their list So one company is a company called bright scope. So bright scope is a startup in San Diego It's grown to be over 70 folks and they provide transparency about 401k plans About the about the performance the fees and about registered advisors So it's help helping companies make better decisions and helping employees make make better decisions so they can achieve Financial security in the retirement Another example on the list is a company called Archimedes. So this is a company that spun out of Kaiser Permanente, I think in 2007 and they're making personalized medicine a reality, right? so when you think about medicine and and Doctors kind of giving you a course of action or talking about interventions What you care about is how this is relevant to you not kind of the general population And so there's there's a number of great companies that are thinking about this Archimedes is One great example of of thinking not about general guidelines But about courses of action that are our individual and relevant to the to the specific patient So this list of 500 is fantastic. It's showing how open data is relevant and fueling a Whole sectors of the economy across different industries And I think we'll hear more about that from the McKinsey folks as well here today The the other thing is that open data is helping us be more Effective in how we work as a government And it's also helping Americans benefit as a result So for example the the US Health and Human Services released a data set this year for the first time showing what hospitals charge for inpatient Stays for the hundred most common kinds of hospital stays and so it reviewed it revealed a massive Variation of hospital charges even within the same metro area So in Birmingham, Alabama for example the average hospital Charge for hip and knee replacement varies from twenty three thousand at one hospital to a hundred and forty one thousand at another Just a few miles away And so the demand for this kind of information is real Within 24 hours of this hospital charge information being made public. It was downloaded over a hundred thousand times Making data open and available also means improving services for the public So when HHS makes a list of health care providers or the VA post a list of regional benefit offices It helps a startup That's making a geolocation app For for Americans to kind of find these services. We're also working To to put to make information more standardized so large search engines can find our data as well For example when you search for a a drug Prescription in a major search engine that kind of information comes up immediately and that's the kind of Thing that's going to help Americans And so while there's a lot of a lot more work to do. I'm really excited about the progress being made on the executive order Let me just tell you a little bit about what what the agencies are are doing So first agencies are create creating a single Enterprise data inventory that is they're required to catalog their data assets in the same way that agencies are required to catalog chairs or tables They're required to catalog their data assets across the agency To for those data assets that are public. They're required to publish a public data listing on their slash data pages You should go check it out So those are all the data assets that are public or could be made public and this gets back to this asymmetric Information challenge that I was talking about earlier and third agencies are adopting new feedback mechanisms both online and in person To help them prioritize kind of which assets they should go after and and make available So I'm pleased to report that we're making great progress over a dozen agencies have launched webpages at agency.gov slash data making it easier for the public to find and understand and use data And that data is both on those pages and it's also harvested into data.gov where all of the US government Open data is listed Agencies are also building important internal processes inside of the agencies To to make data more accessible and and work on how to improve how we interact with The public so let me give you some specific examples just across a couple agencies I see we have department treasure treasury here, but my first example is actually the Department of Education so DOE is offering a suite of new application programming interfaces kind of software Interfaces for the loud software developers To access great tools and data like a solar energy resource finder Vehicle gas mileage estimates and utility rate database among others and they've recently launched the American Energy Data Challenge asking for the public's help in thinking about How to solve America's most pressing Energy challenges and and what's the role of data? The Department of Transportation has made more than 2,000 data sets publicly available and easily accessible That data helps power the National Highway Traffic Safety Administration Safer car app that consumers can use to compare safety ratings locate child safety seat information and track vehicle recalls and Through the data gov platform Which initially launched in 2009 the public can can access tens of thousands of data sets from a wide array of topics From the hospital charge information to credit card complaints weather and climate and so much more And to make it easier for for entrepreneurs We're working to reimagine data gov So I encourage you to check out next dot data gov We're actually Reimagining data gov in a very open and transparent way, so we're using open-source software Something called WordPress and secan to Help build it And the guts of it. It's also an open design. So it's on something called github, which is a Alex knows knows github well So github is is a place for software developers to to share code And so we're making the new data gov completely open and so local governments or or or other governments can can share Contribute ideas and so forth and so We're still just kind of scratching the surface here The White House was White House also developed project open data because we wanted to take the same approach about open open-source software Open design and so project open data, which I encourage you all to Google is is not only where we've put our policy But it's also a free online repository for tools for the agencies it has case studies it has open-source software for them to help implement open data And again cities states foreign governments can learn from us. They can use our designs They can use our our tools or code So I'm really excited to hear from McKinsey today About their report. It's fantastic to see about the the potential of open data and the value that it can create the Over trillion dollars of domestic value and three trillion globally across seven key sectors of the economy And so whether it's it's it's new company and new product creation whether it's kind of increased efficiency of systems Or consumer surplus cost savings all these are kind of fantastic Ways that we can think about the the economic value of open data. I'm sure you'll hear a ton more about it one of my So one of the themes of the of the report And one of my my favorite topics is is how open data can make markets more transparent And so the consumer for consumer financial Protection Bureau is a great example of this, right? So they have something called the consumer complaint database And it allows the public to see what consumers are complaining about and why And when the question in the company in question responds consumers can track their own complaints and regulators can use this This database to inform their work and companies can identify pain points and they're even starting to compete on customer service There's a recent Forbes article about how large banks are now competing on customer service The McKinsey report also talks about this notion kind of a close cousin of open data this notion of my personal data or my data And this is something that the administration has been working hard to further So one example of this is the blue button initiative which started at the Department of Veterans Affairs To make a prescription data available for veterans and it's grown beyond prescription data to think about more more than just that and all health information available back to consumers and so now across both the VA and Tri-Care and other government systems as well as large insurers and payers and other parts of the health care system Over 88 million Americans can can get access to their health care information Due to the blue button initiative And so it becomes a powerful catalyst to motivate Health providers and health plans to to engage the consumers and let them put health care kind of in their hands We've launched my data or personal data initiatives in a number of other sectors the green button initiative for example Gives Americans the ability to to access and download their own energy usage Information so today over 42 million households and businesses Reaching over a hundred million Americans have access to their their own green button energy usage data And if so if you're shopping for for solar panels or if you're thinking about virtual energy audits This kind of thing can help consumers save energy and save on their bills So armed with your own personal information What if it's sitting in the vaults of government or if it's Or if it's from industry consumers can can better understand the medications they're taking they can understand how much energy Their house is using how their child is doing in school And in addition to helping consumers make more informed decisions It's also fuel for for for economic growth and the kind of innovation from entrepreneurs and startups creating new apps and services To help in these markets. So stay tuned for additional federal my data announcements coming in 2014 and so I think regardless of whether we're talking about open data that's available to everyone or whether we're Talking about personal My data that's available safely and securely only to you I think we'll fall short if we're just talking about the supply of data, right? We also have to think about the dialogue with the users of data and that's why the administration has has held a number of data jams these Small workshops with innovators and entrepreneurs About the use of the data and we've held these data paluzas these larger events to to celebrate private sector innovation Using our data and and other sources of data to to kind of advance important national priorities So I can't wait to see the new products and services and companies That that come forth from from the use of open government data. I'm really excited about the about the panel discussion today So, thank you very much Thank You Nick. Thanks to numerica foundation for hosting us and thanks to all of you who some of you might be geeks I know what github are other people Also, just they're interested in the tremendous power of this trend. My name is Michael chew I'm a principal at the McKinsey global Institute I lead some of our firms research on the impact of long-term technology trends And I was one of the co-leaders along with the honor who you'll meet in a moment of some of our research on this topic of open data Excuse me So I thought I just introduced some of the research findings again It's an extensive report. You can download it off the web So, you know, if you want to see all the great details, it's it's freely freely available But maybe I just introduced some of the things that we found as we tried to study what we think is a very Important topic. We've actually been looking at the impact of data for several years now We've led some research on big data, which is just the increasingly complex voluminous Real-time and and diverse types of data which exist We actually think this trend about open data and making data more fluid or liquid is actually an incredibly important one And we'll talk a little bit about why One of the things that what we often point out is that data or the importance of data is not actually new So here's a picture of an important data pioneer. Does anyone recognize who that is? That's Thomas Jefferson, right? Third president of the United States. He made a real estate deal a few years ago You know bought some land who he's in a purchase after purchasing it He actually decided to do a home inspection or something like that So he sent a couple of inspectors out there. You might be familiar with these guys Mary Weather Lewis John Clark and and what and instructed them and through some orders because They were members of the military To go and inspect this land and it's a it's a pretty interesting order you can read it actually on the web He didn't issue it on the web. I don't think but You know, he asked them to look at the floor the fauna He'd asked them to look at the rivers the streams the lakes He wanted to know everything he could about them and he instructed them to to record it in great detail But you know right after that is something pretty interesting He says it recorded in great detail both for yourselves as well as others And I think that second part is an incredibly important realization Which is that as much data as one organization can collect and use and we think that most organizations Governments included can usually capture a tremendous amount more from the value of the data that they already have That that value gets multiplied when more and more people can use it So I think Jefferson was aware as aware of it before As as modern day people are now such as nick But now that we're collecting so much more data we can create so much more value And that's really what we wanted to to investigate So here just a few you know stats and statistics, etc You know you can free to read them but one of the things that we've noted is you know since Jefferson's time This idea of governments particularly but other institutions making data more available more open more liquid Is one that's you know spanning the world So not only the u.s. National government but different levels of government from municipal to state Also, you know large large-scale multinational organizations as well as around the world, right? So we see you know over 40 Governments around the world who have some sort of open data portal Now one of the things that that we're going to illustrate is that we think of open data is really being defined along several spectrum You know you can be you can take a very Narrow view of open data Which is you know represented on the left hand side of this of this chart or you can you can take a more expansive view When we tried to take a slightly more expansive view So those four criteria are as follows first. How many people have access to the data? You can you can you know the most closed data only one person or worn organization has access to it And the most open data is everyone in the world or everyone on the internet has access to it But we think there's a bit of a spectrum there and it's important to recognize that spectrum So take even some of the data that has been made available in health care to from health and human services You know a lot of that data was just kept completely proprietary to that agency And you know through Todd parks and others efforts over the years more and more of that data has been made open or available But not necessarily completely open not all of it's completely open for some of it You have to be a qualified medical researcher But we still think that bringing that data outside the walls of government actually increases its value. So again That's one spectrum Against what you can view the openness of data Secondly, you can look at the degree to which data is machine readable So you know data, which is increasingly machine readable easier for machines to use actually becomes more valuable It becomes easier to use You know, so some data has been released released in proprietary data formats, which you know arguably are still on the web And yet not quite as easy to use whereas other data can easily be extracted turn into a database combined with other data, etc So machinery ability is important cost is also important some some data is completely free or inexpensive free of cost Other data is extremely expensive. Again, that varies along the spectrum And then finally the legal rights or the negotiated rights to reuse analyze Republished data again some data you have completely unrestricted rights to use it some data You have absolutely no rights to use it again. That's a spectrum We just but we do find that as data is Increasingly open can be used by more and more people It can be create more and more value and one of the things that we wanted to do is try to understand How that value gets created before we go there though It's helpful to just understand in the scope of data, you know, Nick already made a little bit of reference to this How we view all the different types of data You know lying right so first of all there's all the data in the world And there's what we've described as big data now big data is not just voluminous data, right? But we've described it and others have described it as a trend where it's increasingly Diverse in its sources and types where it comes from Whether or not it's structured or unstructured data whether or not it's images or video, etc And then the fact that data is becoming increasingly real time You want to be able to deal with it and with great currency? So that's that's this trend we call big data Now a lot of what we've described is open data this increasingly liquid data from the criteria We just showed previously a lot of that actually is going to be big data But some of it will actually not be big data but increasingly open as well If small amounts of data might also be open and then within open data There's the data that's being released by governments And we think that's an incredibly important place We do think that governments are in many cases leaders in creating collecting and distributing open data But we do think other organizations are also able to able to make their their data open And finally, there's another category which nick also made reference to We've used the term my data other people use other terms But that's one in organization makes data available to the people or organizations that it collected it from That has some really important advantages Many cases it's just a transparency advantage It's a great value for me to know what an organization has When an organization has data about me But the organization can also benefit as well because many times when you release open data to the person For whom it's valuable You also give them the ability to correct it or add to it And so that actually increases the value of that data to the organization So it's it's actually a benefit that flows both ways So that being said, what did we do? Well, we tried to understand in a number of different domains in the economy And we tried to look at some of these as being social sector domains We looked at some of these being b to b some of them being b to c so we have education We have transportation consumer products. We have electricity and oil and gas We also looked and leveraged some of our previous work on The impact of increasingly liquid data in the u.s. Healthcare system And then portions of portions of consumer finance which include retail banking insurance And as well as you know some of the aspects of of home finance and and real estate And when we looked across all of these different domains in the economy and we had a global Scope with the exception of healthcare where we looked just at the united states On an annual run rate basis. We found that increasingly open or liquid data Could create over three trillion dollars of value annually Now what's interesting is we view this not only as value that's created in terms of profit the profits to companies New companies or the ability for existing companies to increase their efficiency and effectiveness of operations But we also looked at the value that consumers and individual citizens can derive So take for example price transparency nick mentioned the fact that having more open data actually makes markets more transparent Well, sometimes that value is actually going to accrue to the consumer Perhaps through price transparency where someone's able to you know buy something at the best possible price Well that benefit doesn't necessarily Is not necessarily captured in a traditional gdp metric and yet it's still value that actually gets captured by consumers And we thought it was important to capture that as well So when we looked across all of these different domains a tremendous amount of value can be created And again, it's it's in some cases improving the businesses of existing businesses improving the performance of existing businesses Sometimes it's creating new businesses New lines of business that are based on data And then sometimes it's actually, you know value that gets captured by individuals So let me just give a couple of examples of where we found this value being created So take the education field for instance There we found five different ways in which increasingly open or liquid data can actually create value firstly In improved instruction when you actually have a large body of information about for instance We're seeing this online for instance. We're seeing a lot of more online different types of Courses being placed and one of the things you do when you create a way for a path different pathways You can actually start to analyze the data and figure out if someone takes trigonometry first or algebra first Are they more successful in In calculus are they more successful in statistics being able to pool that data for multiple sources because it becomes more open Allows us to improve instruction. That's just one way that that can happen Secondly matching students to programs actually Having a student be able to tell which program is most appropriate for them. Again, it can be that that ability It's it's a type of Market matching as nick described it can also be enabled by more data being made available about educational institutions About educational outcomes and then and about students themselves Similarly matching students to employment. This is some work that my colleagues at McKinsey have looked at which is, you know Education to employment is a tremendous mismatch many times between The skills and the jobs that are available in the marketplace and more importantly that will be available in the marketplace going forward And the educational programs that students would be best suited to take in order to make sure that they can match The needs that are in the marketplace again being able to provide more transparency more information and more data About the the opportunities that will be there and the skills that are necessary Along with the programs that can be used to get there are extremely valuable And then paying for education similarly much the way that bright scope can provide does provide a tremendous amount of information to an individual Consumer in terms of what they can do to enhance their financial outcomes being able to pay for education As well as another place where open data can be made available and then finally efficient system administration The school systems themselves for example just in procurement The ability to understand how much other institutions are paying for certain goods and services Is it is a fundamental and straightforward way to improve their efficiency? In fact, what we found if you look across all of the three trillion dollars In annual value about a third of that comes through better benchmarking just having transparency into How others with similar Characteristics or either paying for things or in the health care case how people are dealing with their health care So again, if you look across all those five levers We see on the neighborhood of about a trillion dollars per year annually in value that can be unlocked in education Let me just give one other example Which is in consumer products just to you know, take us in a completely different direction Again another five levers one of them is simply improving a product design and manufacturing If you have more information for instance about what people are actually using products for you can actually design better products Products that are more fit for purpose for individual consumers You can also also improve store operations if you have more information Both about the consumers as well as what's going on in the store in the store more more targeted marketing and sales Again, the census information has been used for many years in order to target better But when you actually have more data becoming more available and open You can use that to understand what Individual consumers needs are and then better informed consumption the first three really, you know improve Outcomes for companies But this third one is incredibly important as well If you actually give people more information Sometimes through a mobile app which comes through machine readable data You know number one you could actually have a lot more price transparency And we're starting to see that already where someone can actually tell how much does this item cost in multiple stores But you can actually imagine turning that around where a company might make more information available about its consumer products Say, you know, what types of materials are in that product? Or whether or not child labor was used in the production of that product or where it comes from Again, you can imagine people actually paying more for products because they have that information So I think this you know providing more information about products to consumers Is another potential value for open data where the consumer And the producer can benefit and then finally improving post sales interactions Again, if you know more about how people are using products You can actually improve the customer service and compete on the basis of customer service as well Again, another tremendous amount of value can be captured through open data even in the consumer products Area, so if you want to take a look we we go through a number of different areas in which Open data more increasingly Shared data can be used both by companies as well as individuals and consumers to increase their value Again, we think that Accountability transparency all those things that we've talked about in terms of good government good institutional governance is incredibly important What we think we're adding to the debate is the the value of these types This type of data actually to create real economic value Let me turn it over at this point to my colleague diana talk a little bit about their role of government And and thank you again to the new america foundation for having us here I think what michael has shared with us so far is the results of the report. You'll see outside painstaking detail Domain by domain to try and understand this critical dimension of open data How much value can be unlocked? But of course the value will be both created and captured by the full range of stakeholders in society And we see of course consumers who will be Not only beneficiaries through price transparency through more convenience through better services, but also They must be users. They must be informed users of the data We see also businesses will be able to capture a lot of this value through the introduction of new products and services But they also need to participate in the process of Making data available Processing the data cleaning the data, etc. Etc. So to the media has a critical role to play in informing Society individual stakeholders and otherwise about how this data is being used misused What is missing in the debate and of course ngo's who will harness the power of open data for their own causes But also need to be part of the watchdogs and infrastructure for how this evolves over time Now as we think about all these stakeholders They interface very importantly with the government and we do come back to the discussion That is very apropos for dc about how critical it is for government to Help be a catalyst in its own release of data to help Foster the right kind of discussion and the right kind of dialogue around these issues And you know Michael ended on a note that I think we come back to all the time that Too too much the open data conversation has been Narrowly cast as a transparency and good government good efficiency discussion We think that's very important. We're big fans of that, but we think it's a much bigger conversation around value creation But with the adequate measures of risk and and and oversight Now if you if you think about What those elements are That government needs to play a central role in one is enabling these value levers to actually come about and and become Able to deliver on the promise of the many examples that that michael gave you in the two domains that he touched about We have all seven domains referenced in the in the report itself But very importantly We think government has a critical role to play in managing the risks associated with open data Because there are real risks But also because so long as these risks are perceived risks There will be lots of resistance and barriers to people Adopting all this and we think government has a critical role in convening the stakeholders and helping enable the ecosystem from developing And so when I think about what Todd and nick have done us The government has done to bring together these data paluzas to bring together the sets of interested parties That's critical part because without the ecosystem of software providers without the ecosystem of Businesses and consumers actively demanding this. This is just data sitting on computers. It won't mean anything at all um Now if you think about all the seven domains that we've looked at education transportation energy health care consumer finance consumer products And we've specified the levers at a great level of detail but you can take a lens back out and say the really three categories of Ways in which open data is ultimately enabling value creation One is improving decision making across the board and so whether it's Better decisions about students going to schools better instruction better consumer decisions, etc We think the promise of open data is a lot and and if you think about Government in helping enable that better decision making it. It's both through the promotion of a wide release of data through Helping real-time performance examination Using this data and and being on the forefront of some of the computers that allow Analysts of this analysis of this data and you know, whether it's NOAA or whether it's census or whether it's many of the Organizations within government that are driving these I think government has a critical role To help enable the ecosystem that will make better decisions and create value that way of course A lot of the value comes from all kinds of new offerings new products new services More efficient products and services and we think government has a role here too in helping spark innovation either by bringing the right folks together or by Prizes and things like that that the government has already initiated successfully By helping through its own data Individuals businesses otherwise better segment populations and allow more customized solutions By good regulation that fosters competitiveness as these innovations occur and by Ensuring the supply meets demand in all the ways that supply and demand often get mismatched in markets Finally a critical way the value gets created across all these domains is just improving accountability either accountability of markets or accountability of decisions and And here government has a role to play in increasing transparency and providing good performance benchmarks And actively involving citizens and consumers in the solution process And so as we think about all these domains we think about all these value creation levers We do see a critical role for all stakeholders and government as a big enabler of all that value creation Now of course as as much as you hear our enthusiasm on this where you come out this wide open with the risks and we think here too Um all stakeholders have a stake in managing these risks as does government but just to list some of them and Help with this group of people who are interested start a more Integrated and comprehensive discussion and dialogue on these things that we think are going to be essential to manage in order to Unlock this enormous value. We have documented think about the kind of consumer risks people worry about On top of list is the privacy issues. Certainly in healthcare. This has been an ongoing Discussion with the NSA findings. This is the top of every newspaper discussion And this is really across the whole range of You know who and which data should be collected who and which data should be used What do we do when data is misused who monitors all that and those issues and that framework for understanding Privacy is going to be essential to unlocking a lot of this value So to security all many organizations the government but also private and otherwise are really struggling with Protecting their data and ensuring the the proper Safeguards are in place and here too government has a role in establishing those frameworks Society has a role in debating how we think about The costs and benefits of ensuring that kind of security and ultimately there are set of issues around safety some real some perceived and so people are concerned that If certain data is made available and people it's incorrect They could make decisions that lead to poor safety outcomes. Who's going to be liable on those issues? How do we ensure that the quality of the data leads to the best safety? These are the kinds of consumer risks We hear a lot about and they're very prominent in the press But there are two a number of business risks that are part of this story Transparency a lot of organizations limit the amount of data they make available because they think it may reveal excess information Teachers take real issue with a lot of data being put out there because they think it May cast poorly on their performance in ways that concern them hospitals have real concerns about this How we deal with those transparency issues and how we provide safeguards to the next issue around liability for what this data shows And to whom are Parts of the framework that will be essential to enabling the value And then there's some interesting issues that we think could arise over time around intellectual property that have to do with If you think about the story of patent reform Or the the recent patents On on genes for example for a while the u.s patent office was actually Allowing people to patent genes because it was so difficult to identify genes And now that we've had an entire public infrastructure to Um to identify genes we've overruled that we no longer allow that kind of intellectual property to be harnessed We are likely to see issues like that arise as more and more data become available And we think about who owns the rights around that Of course, there are all kinds of mitigation strategies But a critical debate in society around how we think about these risks with the cost and benefits Are going to be essential for us to properly identify Or de-identify information if you like to invest in the right safeguards and training And to ensure that all stakeholders feel like they have a voice and are heard appropriately As we Complete this discussion of sort of government's role Of course, many of the things I've mentioned is government as a regulator as a rule maker And and that's important because government will be essential to establish a lot of these standards both in the actions It takes and the parameters it sets out there Of course enforcing a lot of those standards one way or the other But a very big part of this is government as a user itself both releasing its own data Applying its own analytics improving programs and putting them out in the public domain And we think very importantly as a catalyst through some of the things that the government has been doing Here in many other countries today convening stakeholders Ensuring educated users and actually investing in a lot of these programs If these roles are taken seriously and we have a public debate that is Sophisticated and involves all stakeholders. We are very excited about the economic value that can be created here So, um, I want to thank you for the time and for your interest We think this is a very exciting topic and to begin that kind of public dialogue I have the pleasure of Introducing alex howard who's going to lead our panel today and as many of you know alex howard is the government 2.0 washington correspondent for No Oh, I'm so sorry. I was giving the wrong note. So I'm going to let you introduce yourself and I apologize for that But um, thank you very much for leading our panel On the live stream This is a really interesting moment to be having this discussion in washington because I think that Hopefully the people who are tuning in here are are connected to each other And also connected to the people who will be bringing up We've got mike flowers. I believe somewhere in the audience here Who is from new york city? and I believe it holds the distinction of being the first chief analytics officer for gotham city and he so he is I think one of the best people situated to speak about the internal value of open data in terms of Applying analytics to get insights. You got someone from zillow and of course i'm missing my program So I don't have the proper introduction for her go Yes, please. Please join us up here Let's see, uh, svenya gudel is that about right the director of economic research at zillow and of course if you're interested in buying Property zillow is a very important area. We've got sasha minrath the home team here vice president numerica and director of open technology institute And me So I'm I guess on the adduck here. So there's a uh, I think An opportunity right off the bat for each of you to speak about what You understood or took away from what we just saw in terms of your own experience the open data What did you see there that relates to what you do professionally? In terms of how you value or consume or create data? I'll start with you. Please. Yes So pleasure to be here. Thank you so much for inviting me to speak Um, I you know, I thought all the points that were made were incredibly good ones I think It's about time that we really take a deeper look at open data big data I think those terms float around a whole lot and people don't always know what to make with them But make of them But I think we all interact with big data and open data on a on a daily basis and um in my job I I certainly deal with it a whole lot at zillow Open data is really at the the core of what we do is as a business And we we truly believe that you have to bring transparency to a marketplace Which is exactly what we did if you think about you know, uh, 10 years ago Eight years ago how you bought real estate was you called up a real estate agent said, hey, can you show me a couple of homes? And how do you shop real estate today you go to zillow? Hopefully or Truly or any other webpage out there and you search for your homes and you do it yourself And then you contact a real estate agent and you say I want you to show me these five homes that I already found so it's it's a complete Difference and how we and how we deal with buying real estate And that's just one example about how open data changed our lives, right? And uh, and at zillow we use local government data. So we we import Facts about homes from from all the counties and that's certainly open data But we also deal with open government data. So, um, I lead our economic research team and we put out research reports all the time that use hamda data or census data or FHFA data in In connection with our own data So we a lot of times morph the data together to produce new insights and help customers understand the marketplace And that really brings me to the last point that I quickly wanted to make I thought nick had an excellent point about data by itself doesn't give you all that much, right? Because if you um If you think about how much data is floating around out there, uh, you know Sometimes people don't know what to do with it. So it's super important to take that data and move it forward and actually interpret And and create knowledge with that data that then consumers can can absorb and use to make better decisions And and create that transparency in a marketplace So I have to go a brief station programming note and invite aman bandari up Aman is the u.s. Director of h it data innovation strategy at murk I first became aware of his work when he was working for the united states government working with todd park on the blue button Another open data initiatives at hhs. So thank you. Aman. It wasn't quite sure if you were here, but now i'm glad to see that you are And for those that don't know I did leave o'reilly media in May of this year and i'm now a fellow at the tau center for digital journalism at columbia university write your research on data driven journalism and Try to speak up for that Fourth pillar that diana spoke to in terms of the media's role and I'd actually add on top of it Not just to hold governments accountable for what data has been released, but actually consume it I think that there's a crucial role for media in this age for people who aren't watching what organizations like bloomberg or Reuters the economist intelligence unit Or this kind of growing wealth of startups. They're all Consuming and creating a huge economic value out of open data And in many cases they're actually creating it themselves now, which is a really fascinating emergence So I wanted to put in my props for that work and also identify myself as a fellow. So with that said Move it over to you mike. You want to field that question as well? Do you still remember the what I threw out there? I think so. Okay. Good to see you again. You too So I mean a lot of what was said is Sort of implicit in everything we do in new york city except we just don't express it, right? Um, or I don't do it very well anyway, but uh, so I'm an open data practitioner at core I am the chief analytics officer. It's a title. I didn't pick but uh, and it certainly didn't result any raises But the where we started out was how can we take the information we have and do something with it to make us better? Uh, and by us at local government, we pick up a trash. We fight crime Get sand ambulances to your house. We regulate your businesses, etc. etc, etc So all of those things that were discussed Resonate with everything we do data is bricks from which we build things, right? Data itself It's data, you know, I mean Good for you for putting it out But the reality is I need to know How to take that information stick it together with something else and then turn it into something that we can allocate taxpayer resources Towards or more effectively, right? So that that absolutely resonated the other thing that really resonated with me was And something that I think Wasn't really deeply discussed but at core is what I care most about in my job, which is the internal transparency Right and I think some of these terms are becoming by repeated usage terms of art but transparency just simply means people sharing information with one another and The thing about government that people always seem to get dinged up about is they think it's some they think it's just a single actor And that's so not true And and and I don't say this is an advocate for government. I say this is someone who is deeply frustrated By the fact that that's not true. I would love an authoritarian state Only for the for the ability to get grab everybody's data everything else we stay with democracy But if I could be you know, if I had a wish it would be to be able to just buy Fiat grab what I wanted to grab to do something with it by in each agency It doesn't work that way, right? Every and and it certainly doesn't work that way at the local level, right? Yeah, the guys in fort mean might disagree on some other part Yeah, I mean they may right but like, you know, the the sanitation department does what it does It picks up trash and it does it extremely effectively The police department does what it does extremely effectively all the controversies notwithstanding at the end of the day They get measured by how many people get murdered in New York City and that number plummeted, right? The fire department, you know, the fire department deals with 20 some odd hundred fires a year And only and I say this only I know it matters that that number is what it is But 80 people roughly die every year in a million buildings in 2,500 fires Right through fire. That's awesome, right? But at the end of the day we can get better and more efficient It's just if we share what the police department knows with the fire department So the fire department knows how to do their their preventative or prophylactic inspections Then we get better, right same goes if the sanitation turns out that if you know There's a lot of debris in front of a place in the sanitation department writes, you know Citations for that That actually is fairly predictive for whether or not bad things are going to happen at that location Or it's at least correlative with and again, I think it's very commonsensical so you know to kind of Wrap this up and what I love about this movement If you can call it a movement is It enables operations people To finally live the dream of understanding what it is we know About our subject matter areas or our problems or our challenges and then address them, right? All the other stuff. That's great. I'm not an academic my wife is she can write papers on it and have fun But the what what I want to do is to live a widgets I'm an operations junkie at core And I really love understanding how we send trash trucks out. I love that stuff. That's fascinating And the more I can know about how we do that and the more I can leverage that on behalf of other stuff we do Right. So at core what I love about the open data concept the movement itself is that it enables us to actually get better In terms of servicing, you know the citizenry and I would hold just you know, I know you're butting me But I would say that those principles apply to any large organization Right You know stove piping and silos and all that stuff. It's not endemic to government It's endemic to any organization that's larger than eight people And so the more we can leverage the information we have Internally as an entity the better off at the end of the day. We're all going to be Thank you very much. Now, I know you can take it. So I'm going to ask you a hard question to follow up Once we get through these out of the gentlemen here, but um, New York City is not releasing its crime data Right, so let's let's come back to that article. Yeah, so, you know, um, this this is this is a crucial issue You know speaking about this particular problem of uh, government releasing an interface this crime map Which city council said they needed to do but not releasing the underlying data for it So if we're going to talk about open data in New York and using the power of data to do these things I think that's ground zero. So we'll come back to that And maybe you can discuss the differences between where you sit in the mayor's office versus where and ypd sits and what They are able to do So you mentioned open data being a movement This is actually a great I think moment to acknowledge that there are a lot of different stakeholders Who are interested in releases of data and they may have different political aims too I don't think there's any uh question that people in this institution in this room Certainly many people watching online understand that collecting and releasing data is profoundly political now Whose data gets collected? How does it get depicted? What are the quality issues in it? Does collection get shut down if the data shows something that's uncomfortable, right? If a law comes through that says you got to publish this data Then do political actors say well, let's stop collecting it or let's obfuscate it Or let's release it in a format that you can only use if you've a proprietary program Or let's set a high cost for it If you look around the world you can see governments reacting to this new kind of openness with all kinds of well, uh fear Right or anger or rage because they're being shown to be something that they're saying they're not in public It's very difficult in these contexts now to have there be a difference between what you say to the people and what the data shows You actually do That is one of the promises at the moment And it is also one of the reasons that you have people who are small government Libertarians and conservatives support open data and also people who are maybe on the progressive or liberal side They both want the same release is one of them wants to make government work better Well, either wants to shrink government because it shows that government is incompetent But that means you might have alignment with very different kinds of ideologies in the same room They might even be here. So, um sasha You understand data is political OTI sure does can you speak about what you saw here and kind of add some context for your work? Sure. So new america spans a number of different programs many of which are Collecting and making available open data. So you have everything from the education program, which is collecting huge amounts of information Not the new not just the u.s. News and world reports kind of info, but in depth info around schooling and the outcomes thereof and different options that people might want to look at You've got folks like peter bergen that's collecting information on things like drone strikes and making available The world's foremost open data set on what's happening in terms of where drone strikes are happening What the outcomes are who's being killed etc? For my part the open technology institute. We have something called measurement lab Which is now the world's largest Open platform for collecting broadband data. We put it under a creative commons zero license i.e. You can do anything you want With these data. It's about as open as open can be that's The utopian kind of future where smart people collect useful information make it publicly available do analysis and open it up for everyone to to to work on themselves to test the hypotheses to to Redo the analyses etc. I think what we're living through is a shift Where for a moment in time people thought open data was going to solve problems and people in power didn't realize Open data could prove a political problem And now what we're seeing is a re-enclosure. It's the release of either unusable data You know, it's like we'll give you all these data in scanned in pdf's call that fopen data fopen data I love it And then you've also got this thing of you've got lots of usable data. It's usable, but it's not useful data It's like 90 000 data sets a few of which are really insightful But most of which are totally innocuous for doing any sort of meaningful questioning of hypotheses That are being pervaded by people in power right that we need x to do y We're like well, is that actually true? We can't test that because they don't make the useful data sets available and so I see the differentiator exactly what you said it's about the innocuous versus disruptive data and what we have is a lot of innocuous data that's being publicly released and very little disruptive data But even if we could get to the disruptive data sets being released publicly, I think there's another fundamental problem Which is where does the locus of control reside in all of these data collection? And we're seeing this right now with nsa surveillance But we're also going to be seeing this increasingly in the internet of things as smart devices permeate into every facet of your life It begs the question should cars report when you're speeding or illegally standing or parked Should your fridge report to your insurance company your guilty pleasures that we're all engaged in and probably should jack up our insurance rates But we would rather not have that be known Should the smart devices in our home inform child protective services when you leave your kid Unsupervised in the kitchen while you go to the restroom And fundamentally what this comes down to is right now you all have zero control Over the data that's being collected and that data is being commoditized and sold from Advertisers from devices etc. Again, you have no control over this And what we're seeing right now today is a further dystopian element of this Which is that your privacy has been commoditized and it can now be sold back to you So just this past week we saw at&t declaring that look if you pay us 30 dollars more a month We won't monitor you It's a market value for privacy. It's 350 dollars a year. There's apparently the price tag They've put on that facet of your privacy. This is unwinnable. Of course right now It's at&t tomorrow will be at&t plus staples the day after that'll be somebody else. You cannot win That mechanism and that is absolutely the trajectory we're on so I view open data as utopian and beautiful In many facets but without privacy protections consumer safeguards an adequate understanding of the tension that it makes available Could become a horrendously authoritarian panoptic State where you have little control over any facet of your privacy. You've been reading the critics So, uh, this is not an abstract thing. Of course if you're watching Where some of these technologies are taking us, you know that say insurance insurance for cars Might say if you carry around this monitor in your car for a month and then give us the data from it We'll give you a break on your insurance That's already happening. You can look it up in New York Times, right So that one example can be extrapolated in all these other directions now While I wouldn't be particularly happy about my smart fridge, you know, emailing etna about The fact I'm having chunky monkey at 2 a.m. In the morning again and again and again. I'm not doing that by the way Then you know to think about what open data means in that context There's no question that it that this is really difficult though, and I think you you spoke to innocuous data sets. Well, let's be fair Some organizations have actually released less innocuous data sets I think one of the most important of those in this town is the health department of health human services now Back in 2010 when they tried to start really pushing on this They did start with less Fraught once about community health indicators. That's where that was the beginning of it Now, however, there is more significant data coming online and nick spoke to that Amon you were part of that from the very beginning And so I know you're at murk now if you want to speak about what murk's doing in this space That's that'd be great. But I'd also like to hear a bit more about this issue of shaking loose data, which actually Does have real juice to it in terms of comparing hospitals Comparing the cost of prices between hospitals that has now been put up on the web Yeah, so, you know, I'll just speak you know kind of on behalf of I don't represent murk but Half of you know myself and the work that I do in the team that that I helped run And you know some of my previous work and I'll Take a different angle to some of this, you know, based upon, you know, where I started With open data actually was 1999 when I first started working at the VA and we used Data from other government agencies to actually cost out every single VA visit and the algorithms that were already available to us Because at the VA you didn't know what anything cost And so, you know, as I think, you know, Michael Chewy said this has got a very, very long history here And I think now all of a sudden we've got this, you know, turbo charging effort That's been taken, you know, taking place for quite some time And I think it's even more significant in the healthcare context given what's happened with health reform in the high tech act And so the way in which I think about it is, you know, very similar to, you know, some of the comments I already made But something that I'm seeing now when I, you know, wear a private sector hat is You know, I probably evaluate, you know, or get to see probably hundreds of companies a year that are focused on health data And the vast majority of them In some form or fashion do use publicly available data There are very few that are based just on publicly available data, but there are some And I think the incredible thing to witness, you know, is, you know, first, uh, that there is this, you know, um, You know, very pervasive use of data coming from many, many open sources that you may just not be aware of But it's there and the smartest companies that I see are integrating that into their platforms And they understand that in the current business context And so there are a couple companies out there, for example, one that that I, you know, that I like a lot in terms of what they're doing It's a company called roadmap They're one of the first companies that I've seen that is using primarily openly publicly available data To understand what the marketplace dynamics are going on with healthcare in this country right now Incredible company and then there are many others that are more on the social good side like direct relief That are using the data, um, to help them with their supply chain efforts and disaster humanitarian type situations And so that's on the on the one side. I think the smartest companies are using this as a competitive advantage And they're that's part of their secret sauce is built in on the other side You know, what gets me really excited and what I'm really interested in is the the data that's out there is becoming a Tool for collaboration for us and for others. And what I mean by that is in, you know, in healthcare in particular The science of using data is, um, is very, um, has is very state and very old If you look at health the healthcare system And you go talk to a health economist or if you go talk to a health services researcher Or you go to talk to somebody who's doing that kind of health policy work and you ask them What has been the fundamental change in how you do your work over the last 20 years? And I guarantee you'll get virtually no answer Because they're using the same methods that they were using 20 years ago And what I'm seeing now with some of this data being made available from the government is that all of a sudden You've got this flood of new talent new methods Coming into the space that didn't exist before from people who come from totally different backgrounds And there are concrete examples of this happening. And so for us, you know, what I do in my day-to-day job Is this is helping us accelerate Looking at the science of how we do our work. So bringing in, you know, engineers computer scientists thinking about bringing machine learning methods To healthcare services that have not hasn't really existed at scale before And so I think, you know, the things that we really focus on we care about is actually the ecosystem around the data And then includes the talent piece of this and that means also speeding up research And maybe doing some of that work faster and cheaper because now this data is more openly available So I can I can give more concrete examples of that But that's a little bit of a different take I think than from the data itself But as you know, what is happening outside of that data and who's coming in to look at that And how is that affecting, you know, at least in healthcare, you know, what's happening there? And that's I think there's some really disruptive forces that are coming into play That that are happening right now List of questions, but I'm gonna kind of riff on what you're saying because I think that gets us into a more interesting dialogue And if you all want to ask questions of each other, you know, have at it We'd like to ask questions of you, actually That's fine, too. You may not get good answers the This question of disruption may be the most, you know, innovative word of 2013 Um, which was to say it's been overused Is actually relevant when it comes to existing industries, particularly around Scarcity of information, right? Now people who follow this space know that there have been repackagers of government data selling that information for decades in DC That is part of the context for this discussion Is that if some of these data sets that have real value are released in an open way There will actually be businesses that are disrupted by it Not just in terms of their business practices as the regulators get more information Putting out into the markets they operate it within but actually in terms of their business models If your job is to buy data clean it up and sell it and government is actually doing that itself What happens to your business now? This is something that you see in many of the entrepreneurs are quite aware of right? What happens if that gets disrupted now similarly What happens if you build a business upon a data set that gets taken offline because the people it describe Are disrupted by it now. I'm thinking of doctors and a Medicare claims database doctors or Say fraud right what happens if you put that online now looking back to your government service Where were there instances where you shared data that then the people it described came back and pushed back on Is there a was there an actual power dynamic that was disrupted by the release of it? You know, I mean I I'd lead that up to my kind of former government colleague is described more but you know I think you know, maybe some of the other panels have had experience in other sectors But I think one of the interesting things to mitigate some of this is what I'm seeing is There's actually a feedback loop now into government and to other places by taking the data that's been made Available but also collecting and crowdsourcing more information to augment that it made that stronger And so hopefully eventually over time, you know, those those crowdsourcing efforts will become much more sophisticated So as to approximate or make better, you know, the that view of the data as a whole So it's a different way of answering your question But I think that's something else to watch here in terms of disruption is people who are taking, you know For example, there's a company. There's lots of companies working with the FDA. One of them is med watcher And they're taking the adverse events reported to the FDA But they're also collecting that from you know from the crowd so to speak from the community And that's augmenting that information. And so I think usually looking at those where there's this dual purpose Where is the win-win for the community win-win for the for the company and then a win for the government here? I mean, I think that makes a much stronger proposition for keeping this data Open and available such that the floodgates are already open. So it's an interesting word available Do you want to speak to that briefly Sasha? See you poised. Yes, so Too often, I think that we crowd source data collection where government has failed In its primary role to act in the best interests of the general public And I'll give you a great example of this. So form 4 7 7 data At the federal communications commission. No one knows what this means, but really what it is is it's we collect as a government Data around where broadband connectivity exists and its costs And then we don't make that available and the reason why we don't make that available is because the telcos have argued That the cost of broadband connectivity i.e. what they advertise general to the general public In like all those flyers and advertisements that you get every day Is proprietary information And because of that we actually have no way to actually meaningfully analyze everything from what's being redlined To where broadband connectivity is and is not And to compound that We then spent 350 million dollars out of the broadband stimulus fund To recreate a national broadband map That wouldn't collect certain key information such that now in top of that We have to crowd source to create an accurate broadband map And so for years we've been telling the fcc. Hey that data that you already collect About publicly available information Make that publicly available It seems so simple and yet we are still battling this out with the federal communications commission. It's just one example It's one example that costs hundreds of millions of dollars And that keeps vital information the cost of connectivity Out of the hands of consumers all across the country And I think you could look in almost any sector and see that same dynamic at play So this is actually a good jump off point For forms If there is a form number number number in washington That can be turned into data And one of the kind of the great Rifts that katie stanton used to work in government came out of google and now is back at twitter talked about Moving from a form for that to an app for that You can only do that if the underlying data is released You can only do that if there's existing political will to push it out If you look at what it's taken to get some of those kinds of data Actually pushed out in a machinery format though. It's often involved lawsuits It's involved people acting as basically as civic activists Right who are putting data on the web often at legal risks to themselves carl malamud if you're watching If you look at public.resource.org has been doing this for years He's responsible more than anyone else for putting the edgar database from the sec online That's what this looked like and just as a historical note This the cost for the servers came from the then ceo novell who then went on to become the chairman of google Right, it's a combination of public and private partnership that put that on now form 990 data Which are the tax violence from nonprofits is a big fight between irs and carl malamud He's also fighting to put open standards on the internet. That's what this actually looks like In practice where someone is fighting over putting informed because there's political juice behind them so Mike data and political juice I'm not worried about getting into a into with you because you're a new yorker and you've got you know Press down there that'll eat you alive almost like london, but not quite So um, there are big fights in new york about data There are probably are always going to be because it's new york and people mix it up But there's also real power involved Can you talk about what it took for you to shake loose data from various places and the arguments you used to do it And then what the costs were involved because people have sometimes suggested that this is free And you and I know that's not true. I mean the cost aside from the fact that I've put on 30 pounds and gone bald in the last four years That's a cost open data for that but now it's so you're absolutely right this But it's it's a it's a melange of actors, right? So you have um By the way, I didn't know that about agar in my old life. I was did financial crime and that thing is a real public service. That's right. Um Nothing that made me enough to not have to work So It's kind of a combination of things one of those things is you have this community out there that seeks information Now I didn't know anything about that when I first took the job I was I was brought in as a logistician So as you start one of the biggest things that mattered in new york city to our activist community was something called Pluto Which is so Perfectly governmentally acronym. I can't even tell you what the what it actually stands for but it's like it's a tax lot map Right and that tax lot map I know about it because it became extremely mission critical for us to be able to take our intelligence and attach it to locations And the and the the rosetta stone for the city Was you know built in large part off pluto now people have now the city charged for pluto and charged a low amount But you know, they charge for it And they did it for years And it was and by the city I mean the department of city planning the department city planning has a division that basically Spends a lot of sweat equity Upkeeping Pluto Um They made a total of 80 grand a year out of it now That's 80 grand out of a budget for the city of roughly 60 billion, right? So it's nothing I mean i'm spending probably 80 grand right now But the In city dollars, but the the fact of the matter is that for them that was a lot of labor, right I didn't know anything about it until somebody I didn't know anything that about the fact that it wasn't free Because i'm in the government it was free to me Until somebody finally raised it to me when I had been elevated to the chief analytics officer role Which also gave me responsibility for the city's open data portal and at that point then I started hearing Oh, when are you going to get pluto out? I was like, it's already out. You know, they they sell it for like 500 Box, you know go ahead and whatever and then they're like no, it should be free So I made a couple of phone calls and then out it went right There was a slight fight And by that fight, I mean there were city lawyers I think that had litigated the issue And the way it went out was we were as what we usually were extremely pragmatic about it They It doesn't go out on the city's open data page because they you know, the law department spent a lot of time saying This is not open data and you know what they're right. It's data product, right? so Because what it is is an algorithm that has a bunch of different sources of information together about the tax lot Right, so what it went out for free on department of city planning's page I'm fine with it's out You know like the letter of the law didn't actually mandate that pluto be called open data even though it's extremely important To understand all the other data that's out in new york city So if the lawyers were were okay with us putting it out for free on dcp's page Fine So the lesson as usual is to ask your counsel How do I do this? Not whether you can do this right and and you know, I am a lawyer and which is I'm a recovering lawyer and the Lawyers most of the time are actually your biggest challenge Not the politicians and not the real estate industry in new york, which is our version of like the military industrial complex and You know, it's it's the it's the lawyers and the lawyers It's not because they're evil. It's because they're terrified of getting sued and rightly so we get sued constantly so What they're basically trying to do is make sure that we don't get sued So every question is like oh, we'll put this out for oh, we're gonna get sued, you know, you're gonna get sued anyway So we're gonna get well that's my argument right and then and I'm not shy of court So what we ended up doing was kind of saying to them get me from a to b I don't want to hear that we can't do b because we're gonna get sued right That's the definition of disruptive data. I mean if you're gonna put data out there most likely you will Someone will be upset or you will get sued and that's how how we get things started right a lot of times You don't just put something out and everyone's happy about it They're always stakeholders who don't like it like take for example Yelp they now on their webpage when you look up restaurants Of recommendations they have on there How is that restaurant rated in terms of health violations and they give you the violations and they give you the health rating I'm sure the restaurants aren't very pleased with that Especially if they're towards the bottom of the list, but it's amazing data to have for consumers right so Or take nondisclosure states and of course I have to bring up a real estate example But you know there there are places like texas who don't tell you how much a house sold for officially every other state or Every other disclosure state will tell you how much a house sold for But texas will tell you what the mortgage on the house was but not how much the house actually sold for So it's really bizarre and of course some people like this and other people don't like and I feel like This happens all the time and data that When you put something out some stakeholders will be happy and others won't and That's what happens with disruptive data or that's how we thrive and that's how we push the the process forward So it is a political will issue, right? That's how you overcome These arguments because at the end of the day if it's legal to do it. That's all I care about Give me the am I is it legal for us to do x of y right and if it is Then I'm willing to leverage the power of the city hall email address To to get it out is it legal to release a health inspection data in the city of new york So that it can be ingested by yelp because let's be clear that health inspection data That's coming from cities now that and they're trying to standardize that it already is already out there It's out. All right, it's been out for a while and the restaurants were not happy, you know, they were very unhappy Uh, and and we don't want to dismiss that right like there's a there's a balancing of equities here restaurants There's 20 some odd thousand of them. That's 20 some odd thousand small businesses for the most part You know the ones who actually have the best health grades are like, you know Subway and mcdonald's and you know the chains Because they have the overhead to go in and do whatever it is they need to do to make the place You know spotless, right the ones that actually get in the most trouble are the mom and pops And that actually holds for most places the pharmacies that get in the most trouble are the mom and pops These are a lot of places that just don't have the overhead right to live up to the regulatory standards And so when we're talking about, you know, oh, they're going to get upset I mean these are these are small business people that we don't want to dismiss They generate most of our job growth in in New York City So let's talk a little bit about who gets upset zillo's gotten into the you know news in some directions here because You've done what often data consumers and presenters Do in these ecosystems and extrapolate on all these areas Which is to say make something that was formerly tacit quite public In this case i'm thinking about When foreclosures pop up now previously in the real estate market only people who are in the market right the brokers people going back and forth Between the listings they knew about it But the market didn't and you all got some of that data and you put it on the internet And now people can see when a house gets foreclosed upon and act upon it now There are people who are social justice advocates who felt like that was a bad use of open data You all however were using it to improve your product and you know improve something that some of your users might have wanted What kind of pushback did you receive over that and what did people try to You know close down or obfuscate the data that you're using For that part of the product, you know, it's that's a great question. I I feel like we've actually Had to cross that that threshold with a lot of data products what we put out I mean when we first started putting out this estimate, I'm sure some of you out there hate your's estimate too um People will like oh my god, take it off and the same with foreclosure data. There were certainly Parties out there that said take it off, but the fact is it's already out there. It's already public data You can look up if someone is Past due on their mortgage payment or has started the foreclosure process It's actually just kind of tough to find and investors do this all the time They go and they find out, you know, what homes are foreclosed on then they do research on that home And then they try to bid for for it on the court steps to try to actually get it at a really cheap price We thought hey, wouldn't it be grand if we made this data available to consumers Who oftentimes don't know where to find this data so they can be competitive together with the investors and actually find out this this Data themselves and then buy a foreclosed home and You know, some people didn't like it and other people did like it So it's I mean it's we are careful about how we display it on zillow and we we take great precautions If someone contacts us and says please take this down. It's incorrect data. We do so So we we are very very conscious of that and I think that brings up a good point that It's really important to have that back and forth So that you know, you inform consumers and you don't just say here's the data and do with it What you want but you explain to them what it means and if it's wrong you address that issue and you take it down So we try to do the best we can there But it's also been really interesting because that's actually created a whole lot of Value to other people too, which kind of goes along with today's theme Where then companies companies popped up that said now that this is really available to consumers And they're getting more and more into this and I think you know 2012 2013 were years in the real estate market Where everyone was really hyped up on buying foreclosures and flipping them or just investing in them turning them into rentals But it's really hard to kind of find out you aren't actually allowed to walk through a Foreclosed home before buying at auction So it's really tough to kind of see what the house looks like on the inside and you have to make sure you know What sort of back taxes are on that house? So it's it's you know these companies popped up that said hey, I've done this You know gazillion one times let me help you you just have to pay me a flat fee Or not flat depending on the service they're providing and I will guide you through this not so straightforward process of buying a foreclosed home Or an r.io or at home at auction and and you know we created value with that the consumer was happy because they had someone to help them through this process and Certainly companies made money off of it. So it's it's one of those examples where Data got pushed through and it actually helped people make better decisions and created some value along the way So this future where these kinds of conflicts are happening is going to get more intense If you look back at the past year, you can see two examples that should be instructive One is the gun data map who remembers that the new york Now this is up. These are public records. These are handgun permits from new york state that a paper been mapped out Now that probably made a big mistake and I think an ethical one in putting names and addresses on that which made people searchable in the way that they weren't before And it increased personal risk for them if they were of interest to somebody else Now what subsequently happened is new york state changed its law and said this kind of thing shouldn't be available Under foya to a media organization That's a bad outcome for all the media in new york state in terms of restricting access to something that was previously public But there was a use of it that created such a public furor That it caused in a shift in the actual legal underpinnings under which the public had a right to access something Mugshots have become I think a locus point for this as well another kind of public record that often media think has public interest value Unfortunately, there are sites that emerge which monetize them speaking of the economic value of open data such that they then Effectively held someone over a barrel to pay to get it taken down from the mugshot site which often ranked very high in search Now speaking of the ecosystem google adjusted its algorithm and said we're not going to list your image high If you search for a name from one of these sites anymore, which changes the economic value creation from that standpoint But we can look at those two examples and see that multiple actors might use public records in a way that weren't anticipated And then there needs to be some combination of media NGOs tech companies which wield enormous power as processors to change the dynamic for people who didn't have power in that situation Which is to say people whose information was described Now that's a major concern around health data The most so-called mosaic effect goes into you know, uh this discussion. Can you speak to some of The tensions around how you released health data and the consequences of doing it badly In terms of hhs's approach and murk as well because you all have You know everybody I've spoken all of my my colleagues whether it's public sector private sector I mean this is really one of the the top most things on their mind is this exact issue And so it's always at the forefront of any decision that's made no matter what you're doing Um, whether you're an academic you're in the private sector public sector And so I think healthcare in some ways Has been very careful about what types of data people are releasing and how they're using that And so I think there's some you know good examples of this and you know when you know, we launched You know the open data plan from from hhs and how when we started there we called it the community health data initiative That's how it started and that was because we were releasing data about Communities as opposed to individuals so it really depends on the type of data you're talking about And can you have that mosaic effect of the community level you can still have it at the community level But it's a much safer way to do that and I think now by you know releasing more data You also know um who the actors are in the ecosystem because people are you know coming forward and talking about that Um, and so I think in healthcare it's always going to be a huge issue But it's one of those um tensions that that's never going to go away Um, and I think it depends on what is the intention of of what you're you're using that data for So I mean, I think it's it's a huge it's a huge issue going forward. Um, but but I think what's unfortunate is That it is treated evenly across The entire set of actors who use this data So let me give you a really quick example of that pushback and tying into mike's point about political will You know back in in 2006 We had the largest change to healthcare in this country Since medicare which was launched of of medicare part d and all of a sudden we had data on 25 million americans overnight It took us three years to get that data out to researchers and to people inside the agency at cms So the political will at the time and the lawyers at the time the staff inside of cms I was one of them at the time the interpretation was that we ourselves could not look at that data for public health reasons And so I think that you know while we're talking about, you know, the the value of the economy here I think that the research community academic community has got Painted in the same light as all of the other actors in the ecosystem And I think that's really unfortunate and I think that it'd be Um, it kind of behooves people in this community to talk more about Researchers getting access to more specialized types of data because ultimately that fuels the downstream effects of what some businesses can use to You know power off the data that they're getting and so I think that's another really interesting example push back there And being treated, you know, kind of the same as anybody else who would come in You know if you were a pharmaceutical company or an insurer coming in to get the data versus an academic researcher And I think that's where we need to get this discussion much more sophisticated And we're still in the very early days of how to deal with some of these privacy issues And who should it apply apply to? Thank you for that So one of the things that this government shutdown taught the nation all of a sudden is that government data Can get turned off Lo and behold data.gov went down The census api got turned off That changed I think the context for the discussions around these areas. It also I think Showed people where businesses did or did not need that data if you can turn something off and no one squawks What does that tell you? So there are two things that I think took away from that one is that having Immediate access to data was not enough of an economic impact within those three weeks Except for perhaps the BLS data that was the one data set the Bureau of Labor Statistics and the jobs numbers that I saw most journalists Kovach about because they didn't know what to do with themselves on a Friday where they were supposed to come out And so they're you know kind of speculation wasn't wasn't sure what to what to do now Of course, that's a great example of economic value creation people are then making decisions for their businesses for other things That all of a sudden don't have an indicator to include Zillow uses census data though, right? How often do you crawl it? What kind of impact would a shutdown have upon you if it were protracted or if census decided that For whatever reason commercial entities weren't supposed to have it So we we actually noticed we couldn't download our usual data sets that we usually download But I mean for us. It was okay. We we download it for research purposes So instead of going out that week we pushed our report back two weeks and had to wait for the data to become available again So we could download it. Um, it wasn't It wasn't great, but it was all right to me actually government data is incredibly reliable and I don't I think it's It comes up a lot more because everyone was talking about the government shutdown and therefore it was Very heated discussion that this data wasn't available anymore But if you think about it, this happens all the time, right? This is that a few, you know Cloud servers go down and disrupt businesses. So to me government data actually is extremely reliable I know it, you know Friday of this month The jobs numbers will come out or I know that this will happen unless A government shutdown. Let's let's be honest doesn't happen all that often and It's hoping I'm the senate made a budget deal. So theoretically a year. That's right Hopefully won't happen too often But you know what? I mean, I think it's it's usually that's really reliable data other data I don't consider as reliable. So while it had an impact on us, I agree with you Three weeks wasn't enough to really make us win too much about it So one of the things that comes up in my discussions with entrepreneurs is this issue, right? So wait a second if they can take this offline, how do I base my business upon this? Um, you know, how can I uh, you know, really a deal with that risk? You know if we're talking about the economic side, not the accountability and transparency side, not the the service side Although that's clearly very important, too. If you take something offline, um, how could you as an entrepreneur? Can you do that? Um, do you need a service level agreement, you know to some extent now? New York City has got their the big apps challenge You all are trying to take this approach of convening the city entrepreneurs tapping into your tech sector to say come use our stuff What kind of uh, you know guarantees or agreements do you put out there as? Guy in charge of that portal that we'll keep making this stuff available to you even if Your business ticks someone off and they want you shut down Or even if there are costs of you know, we discover in cleaning up and providing this to you that we're not sure of Are balanced by the value creation yet because you know, you're still getting going So as a threshold matter in New York never shuts down so Um, we're not the feds and I say that with a measure of pride, but the reality is Is a number of interlocking reliances on this information that will make that true There are certainly instances where our website has gone down I know it goes down on a regular basis overnight for maintenance, right and we post that It's not a 24-hour Operation yet There are parts of it that are becoming 24 hours as we the government start to rely On some of that data the the taxi and limousine commission Runs a lot of their license vetting now off of our open data portal, which you know, it's going to end up saving us a ton of money Um, it's smart, but that also means we're now converting something that was sort of seen as a hobby Right or is okay. I guess we have to do this right to a business reliance tool And if we're conducting government business off of our open data portal, then it'll get what it needs to be a 24-hour operation And then finally the more businesses that are out there that are using it Will also form a powerful enough constituency to mandate that the resources be allocated So that it remain a 24-7 operation, right? So it's not just there's no silver bullets Just like with anything. There's no silver bullets with this stuff. It's the more I can make The more we can make the the ecosystem as the work gets thrown out I think it's a good one, right because it's it's really true. There's a lot of interconnectedness out there Um, so the the business Part of using open data if we get the more we can incent them to grow the more Our open data portal becomes reliable the more government relies on open data the more it becomes the more reliable becomes One of the areas where nyc.gov got crunched was before hurricane sandy Can you speak to the role that that releasing open data about the evacuation map or flooding zones played in getting that out to the public When that website went down? So nyc.gov. So what you're watching actually doing the last dozen years of the bloomberg administration is a convergence over our information approach as technology approaches operations approaches um Over time, right? They are starting to converge now nyc.gov is not was not actually considered and is still not considered open data Right. It's information impartment to the public So Yeah, I'm sure they would but they'll scrape those pages and make them in the day Go nuts, you know, I mean we want them to but they but I'm talking about inside government How are we dealing with this? So inside new york city government? It wasn't considered that it was considered all the way for you know people to come in Open the open data portal has been viewed and this is sort of the danger Of constantly focusing on open data as a data issue Is it's it's sort of seen or it was seen until about a year ago is primarily like a term of art Okay, it's data. It's ones and zeros. It's in a csv. It goes on the open data portal, right nyc.gov is more like Insight about where to go what to do who to call, right? If you're going to do whatever We're starting to see those things come together, right? So now because what sandy taught us sandy taught us a lot But one of the things that sandy taught us was that nyc.gov needs to also be a 24 seven operation And it needs to be more aligned with how we view open information in the government, right? So I think we learned I learned a great lesson. We all learned a great lesson from it Rachel how this who's our chief digital officer was the business owner for nyc.gov and she really spearheaded this this This streamlining and making more robust Not only the nyc.gov but also how we're leveraging, you know, social media and other outlets to let us To let the citizenry know where to go as the waters were rising and I think What you're really seeing what's sandy what things like sandy what they're what they're really Useful for if such a thing can be applied to a catastrophic storm Is that it really crystallizes very quickly where we're not doing a good job of aligning all of our efforts in a single area And one of those was information So the the story there if you haven't looked it up is the wnyc Which is new york city's npr news station I've got to drop down my baritone all of a sudden talking about them They took open data from New york city's portal about of the evacuation map and flooding zones and put it on to a map And that map reached 10 times as many people as it would have if it had been sitting on nyc.gov Which by the way was getting hammered because people were trying to get on to it to see what was going on now Unfortunately, new york city has got some of the same scalability problems with its data center that certain federal websites do And we can talk about cloud and scaling in a different conversation And as a result the release of that data actually got Information unto the phones and tablets and devices of new yorkers who needed it in the moment And it's a great opportunity to jump off to the conversation that 21st century government needs to have Which is to start thinking about web services, not websites Everyone wants to talk about assigning your website and not as much about the underlying data for it This is something that you know, you guys are going to get beat up for under the blasio if you don't release that crime data I'm going in two weeks. All right I'm not getting beat up by anyone Whoever whoever follows you right because releasing a map without releasing the underlying data is no longer sufficient anymore It's just not so I think we have uh, uh, what four minutes So, uh, I'm going to give each one of you an opportunity to think a little bit bigger About where we're going because you all have perspective in the space You've all been in the trenches. You've dealt with upset users. You've dealt with data owners You've dealt with format challenges. You've dealt with politics Um, what are the biggest successes and challenges in the space and how will politics and power Play a role in future outcomes. What worries you or excites you the most? I'll give you a shot first and then move on down Really alex four minutes for that one. Uh, no actually each one of you get one. Okay. That makes more sense Um, so I think uh, I'm super excited to be in this space. I think data Wants to be free or if I were to say that correctly data want to be free, which sounds really weird And it's going to be free And I think we're we're we're going to evolve in this on this front and we'll Experience this tug of war where some people like it some people don't but eventually data will become free And I think business models are that are based on monetizing data itself will probably become antiquated at some point We see more and more businesses pop up that make data freely available and don't monetize the data But monetize say a marketplace much like zilla. We don't charge consumers We get our money through advertising and so I think more and more of this will will happen And I think it's a good thing you create transparency and you allow Um consumers to better educate themselves. You're spreading more knowledge and that's always a good thing Now what worries me is we do of course have to have some sort of structure around this We need to make sure privacy is in place. We need to make sure No stakeholders are unduly hurts And and so there is certainly a role for government to be involved to structure this release of data We just have to be really careful that there's no one out there that says this data is useful this data isn't useful because They're always going to be people that find all sorts of data useful So let's let's keep that in mind and make as much data available as possible and people will find some use for it And zilla will too Yes Mike um, I think the thing um So so what I wish for my success or if there is one, um Is that They continue to democratize it and by democrat by democratize the way we use our data and by that I mean not just you know pushing it out to the public, which I think is great But and you know it ends up being like this narrow part of the public um, I think So it's what I wanted by democratized. We need to make it more Insightful for people. I that thing you mentioned by him by wmyc. That thing was awesome. That's a great mat They did a great job. That was useful. We steered people towards it, you know as our own servers got overloaded Um, that was good stuff. I'd like to see more of that the the thing I'm most concerned about I think is What I'm concerned about in any job, right? I'm bad managers. Um, I think uh These aren't technology challenges. These aren't legal challenges. These are cultural and political challenges And those are only resolved by decisive and in for intelligent leadership and um, one bad manager can muck up 10 000 employees, you know with citizens, so I think That's that if I had a large concern and it's a concern that I would have over any enterprise It's that we're just the managers aren't up to the challenge. That's a challenge throughout government. I think we can say Sasha I think open data is a normative question for civil society And by that what I mean is how we use this How we allow it to be used Is an open question that we haven't even begun to address and what I see is not a positive future but one where Actuarial redlining and empirical discrimination based upon profile analysis replaces What is currently illegal business practice today? We are heading in that direction And the problem that I see is that most of the key decision makers lack Any semblance of technical acumen? That this panel here represents a good portion of the open data acumen available in dc today And that's a problem So tomorrow we're going to lose like a good percentage of our technical acumen when they all go up But no seriously though This is a real problem because the questions aren't even being asked in these spaces Of how do we balance this? How do we protect people? How do we ensure that this? positive Potential future isn't undermined by some faustian bargain. We didn't even realize we had made That completely undermines For certain constituencies especially The positive outcomes that open data make possible I can't be as poetic as that but but definitely echo You know, I think what what everybody has said I'm really concerned about some of the technical acumen as well as the leadership pieces On the positive side, I think the biggest success has been there's been an enormous change in culture In in the communities and you know that are pushing open data at least, you know In dc and other places across the world and there are real assets and products in place now That will hopefully drive that forward. So it's not just about culture and leadership and policy But there's real product that's there. So I think that's been to me kind of the enormous success That that will hopefully keep this this all going The part that I'm enormously concerned about besides what was already mentioned is You know, when will the open data community no longer be niche just to the people who know and talk about open data But it starts to affect the people who are traditionalists and whatever sector you're in And so I think you know in healthcare in particular Where that will be enormously important is not people who are just building businesses and niche businesses And who aren't doing it for you know, certain public health reasons But when the traditionalists who are crafting and making policy when they start to Commingle and work together with people in the open data ecosystem Because at some point there will be dissonance between what people will be finding from using open data And what the traditional policy makers are using to make policy And I think having that feedback in is a big concern of mine Is when will the traditionalists basically get on board because they're not there yet So there's a phrase that's commonly attributed to Francis Bacon Usually comes up though in the context of a fellow named Thomas Hobbes Who's one of the Enlightenment philosophers that I think The founding father we heard about Thomas Jefferson read as they put together the constructs that underpin How we think about democracy in the United States It's Cientia, Potentia est This knowledge is power Now I don't know if datum Translates in quite the same way Because there's all of these steps that you take to go from raw data to information To knowledge and then to that really precious thing, which is wisdom And wisdom and knowledge have real value in society That is where the economics come from around the space if you have something in scarcity And that is, you know, if you look through this whole chain of data being released That's where you actually can get into Somebody's life in a very meaningful way help them in a meaningful way or destroy them in a meaningful way If we look at where redlining might Move in the 21st century If you look at where Data wasn't shut off. However, though in the shutdown whether data global positioning system data You can see where certain kinds of knowledge that are being given to people in the palms of their hands wearing when they needed Is now woven into the fabric of daily life And that if you look at all these different sectors, you can see where a similar kind of map For an endeavor might grow up that placed you as a little glowing glowing blue dot upon it That helps you trace Not only where other people are within it, but the mile posts The traffic issues The Kinds of decisions that you need to make at that time The question of whether that kind of future evolves or something more dystopian I think is the one that governments and NGOs and media and everyone else who's involved in the space Are going to have to work together to grapple with and make better rules around Um, I'm really, uh, I think heartened to see them emerging more into the public policy debate because as you said This has often been a smaller group of people But the fact is is that these kinds of releases affect everybody And uh, if you look at the best versions of open data releases They're the ones that make their ways into the hands of citizens using the interfaces They already use on a daily basis if you google aspirin you're going to see NIH data in those search results You'll see also economic development indicators from the world bank because they've released them into the place where citizens are already looking So the question now is who will be those kinds of information brokers who ingest data in the 21st century Some of them will be existing. Some of them will be new Um, I'm sure that uh, some of these people here will be involved in them I'm advising them. Maybe starting something up. I don't know what you're doing after this mic Um, but uh, I'm certainly hopeful that new america will continue to put focus upon this I know that mckinsey will continue to follow up Um, and it's my hope that everyone here and everyone listening online also stays engaged in the conversation Um, because it actually does matter people in government really do need to hear from you So, uh, thanks for inviting me here. Thanks to everyone here. Thanks to everyone online. Keep the conversation going