 For the introduction of the panel, I'm going to turn it over to the moderator who is Chris Calabrese and really happy to have Chris here. Chris is actually stepping in because of a family emergency for Laura Moy from the Open Technology Institute at New America. But we are quite honored to have Chris available to do so. I mean Chris is the Vice President for Policy at the Center for Democracy and Technology. He's been a long time been an advocate for privacy protections, for internet openness, for limits on government surveillance, and for fostering the responsible use of new technologies. Before working at the Center for Democracy and Technology, Chris worked for a very long time at the ACLU. So I'm going to just turn it right over to Chris and this is another panel that's got a lot of information and I hope you guys can stay for it. Cool. Well, thank you very much. So I'm excited to be here. CDT works a lot on two of the three areas that intersect this panel, which are the use of data and on discrimination. We do not work at all candidly on competition policy. So I am hoping to be someone who will learn a lot from our panel and maybe I can be a little bit of the voice of the layman. So if I ask stupid questions that you folks already know the answers to, I apologize. I'm going to try to keep them moderately informed. I'm going to be very quick with our introductions because we don't have a lot of time and I know you want to hear from them, not me. I'll introduce folks in the order that they're going to talk. We're going to do five to seven to maybe 10 minutes from each of the folks. We'll try to leave some time for chatting between us and then for questions from the audience. Jeff Larson, a reporter for ProPublica. Ashkan Soltani, technologist about town and former FTCer. I'm sorry, actually we're going to go to Maurice second, but Maurice, law professor at the University of Tennessee and former DOJ. And then at the end, Lisa Gormson, director of the Competition Law Forum at the British Institute for International and Comparative Law. So with that, why don't I turn it over to Jeff? Hi everybody. I'm Jeff Larson. I'm the data editor at ProPublica and we're going to switch gears a little bit from monopoly to talk about data analysis and the effect that algorithms have on our daily lives. For the last year, we've been investigating algorithms at ProPublica and we're particularly interested in the ways in which these algorithms fail. And when these algorithms fail, when they make bad predictions, which groups are affected? I'm going to highlight one, I'm going to highlight one high stakes example of this type of failure. Earlier this month, we published an article called Machine Bias about law enforcement's use of risk scores that try and predict a person's likelihood to commit another crime. So when you're picked up, they give you sort of a test. There are a number of these scores in use around the country but we focused on an algorithm called Compass created by a company called Northpoint. We looked at scores that were given to people in Broward County, Florida before their pre-trial release hearing. So you get picked up before you go to trial, they figure out if you can go home or how much bond you have to pay. Everyone arrested in Broward County is given a test that can contain as many as 137 questions like your high school GPA and whether or not you agree with the statement a hungry person has a right to steal to eat. And Northpoint software runs these scores through a statistical model called a regression and that regression spits out a score low, medium or high risk whether or not you have a low, medium or high risk to commit another crime. We found that these scores are only correct around 60% of the time. So slightly more than a coin flip. And when they're trying to predict violent recidivism, they're correct around 20% of the time. In other words, the general recidivism test is wrong 40% of the time. And the violent recidivism test is wrong a staggering 80% of the time. It means only one in five defendants is predicted correctly for their future violent recidivism, which obviously is a serious problem in a criminal justice context. Now with any decision you make, there's one way to be right and two ways to be wrong. For example, the compass algorithm can only be right when it correctly predicts that someone will go back to jail. So within a window of two years, whether or not someone committed another crime, but it can be wrong in two ways. Either it predicts someone as safe and they go on to commit another crime, or troubling more troublingly in criminal justice, it predicts someone is dangerous and they do not commit another crime. Essentially labeling someone as guilty when they're innocent before trial. It gets worse. Black defendants were twice as likely to be in that latter category than white defendants. They were twice as likely to be labeled dangerous and not be than they actually are. And it made the opposite mistake with white defendants. White defendants were twice as likely to be classified as safe and then go on to commit another crime as black defendants. And Broward, black defendants had longer criminal histories and did recidivate a slightly higher rate than white defendants. So we ran a statistical test to correct for those differences. And black defendants were still 45 percent more likely to receive a higher score. So even correcting for your criminal history, correcting for your age and your gender, if you were an African American, you were going to get a score that was 45 percent higher than an average white person, just based on those other questions that you answered, one of the 137 questions. Now we don't know what actually goes into this algorithm the company wouldn't tell us, but we were able to test the outputs to see this disparity. One of the people in our story, James Revelli, when we showed him his score, we showed him, we said, you have a low score. And he said, that's very surprising. I just got off of a five-year bid. And after that, I did two more crimes. James Revelli is a white man. One other thing that didn't make it into the article, but I do want to share with everybody, this algorithm also didn't work across genders. So a high-risk woman was only as risky as a medium risk man. So again, we're saying people are dangerous when they actually aren't. And I think in a global context, testing algorithms this way in the way that they fail is not something that a lot of people do. And it's certainly not something that the company did. They actually inflated their score. They only paid attention to the amount of time that it was correct. Their validation tests said it was correct 70% of the time. So it's a very quick overview of our story, kind of a troubling example. Thanks, guys. Thank you very much, Jeff. I'm going to turn it over to Maurice now. All right. Well, many thanks, Barry, for inviting me to this conference. Antitrust law is often tested when it's applied to new industries and new business models. One of the things you always hear is, well, antitrust doesn't really apply, right? Because that applies to the old economy. So one issue is whether or not antitrust is ready in the era of big data and big analytics. And Alan Grinness and I are working on several projects with the Data Competition Institute. The first, what I want to talk about is our book, Big Data and Competition Policy. Oxford University Press has it already out in the U.K. It should be coming out in the U.S. in August. And then the second project that I'm working on with Ariel Esraqui is called Virtual Competition, the Promise and Perils of an Algorithm-Driven Economy. Harvard University Press will publish it this fall. And we had several goals in mind with these projects. The first thing was to understand what is big data, right? What are the four Vs that characterize big data, the volume, variety, velocity, and value of our personal data? The second was to better understand the competitive significance of big data and big analytics, like the algorithm-driven economy, and how that's motivating now companies to obtain a competitive advantage. And then the third was to see whether or not market forces will necessarily protect our interests if left alone. And what we find is that there's a disconnect, that we have many more items that are free today online, free items, free services, free apps, and the like. But fewer customers feel in control over their data, and there's an increasing concern over their privacy interests, a hopelessness about being able to protect it. So why isn't privacy competition more robust? And one explanation that we explore involves market power. So right from the beginning, we want to say that big data and big analytics is neither good, bad, nor neutral. It really depends on several factors. It depends on how firms employ these technologies, the firm's incentives relative to our incentives, and then the market characteristics. And big data and big analytics can promote a competitive online environment in which we all benefit, but we cannot always assume that we will always benefit. That some online markets, and that's why we call it virtual competition, may be subject to ordinary free market forces. We may think that we're just ordinary consumers making ordinary purchases that are rather unremarkable, but there's only the illusion of competition. We have no idea the extent to which that we're being exploited. And we can be exploited several ways in this new economy. And the question then is are antitrust tools ready for them? The first is when computers collude. So as companies are increasingly shifting from humans to self-learning algorithms, there will be new types of collusion that are available. And these types of collusions will be covering industries that weren't susceptible to collusion before and will be much more durable. So the first concern we have is computers colluding. The second is almost perfect behavioral discrimination. And here firms track you both online and offline. They collect data about you. They develop a profile about you. And then they target you with personalized ads to induce you to buy. And we call it behavioral discrimination because there are two components to it. The first is price discrimination. That's basically algorithms identifying your reservation price, the most that you're willing to pay. But the second component is a behavioral component, is to induce you to buy things that you ordinarily wouldn't have thought you needed in the first thing. So basically getting you to pay more for items that you didn't think you needed. And a third way you could be harmed is what we call a frenemy dynamic. And Senator Warren touched on this in her lunch speech. And here frenemies can be firms that have an unusual relationship in that they're both friends and enemies. One Wall Street analyst I think said it really nicely is that apps are worth millions, platforms are worth billions. And who has the power going forward in this big data economy? It's going to be the platforms. And the dominant platforms right now are Google and Apple with their mobile system. And that's going to involve with the Internet of Things and the rise of personal assistance. And here you can see the emergence of a frenemy relationship. The apps and the super platform are friends in that they cooperate in being able to track you, gather data about you to foster behavioral discrimination. But then they also compete among themselves over who gets the spoils. So if we were to look at this from another perspective, it would be almost as if a dental alliance cooperating to how they're going to then corner the gazelle and then who gets them the choice cut. And it's the super platform because they have the power and their power is enhanced. So the big picture here is our books aren't just about big data and big analytics. It's really about some of the weaknesses in antitrust enforcement over the past 35 years. And I just want to touch on a few of them in conclusion. So one weakness we already heard this through from some of the speakers in the previous panel and earlier this morning is antitrust price centric analysis. The antitrust has developed very good tools to measure the effects of mergers on prices and narrowly defined markets. And so what is measurable has become increasingly important. But things that are important but aren't measurable have been downgraded such as privacy protection, quality, and even to a certain extent innovation. And antitrust tools don't work very well when products and services are free because you can't you know you can't really ask well what would be a small but significant non-transitory increase in price when the price is zero. A second concern is the Chicago School of Economics that you heard from Burt Ford also from Senator Warren and here it's really the concern over false positives rather than false negatives. There was this belief about self-correcting markets that many mergers create efficiencies so we really should have a light touch antitrust. And the third weakness is the basic collapse of our monopolization laws over the past eight years. There haven't really been any cases brought of note over the past eight years. And this is especially problematic as the last panel brought up in these data-driven markets where scale and scope and network effects are very important. And what you basically see in these types of markets is that the big get bigger until they dominate the industry. So in conclusion, antitrust has lost its way. We're hearing now from critics on both the right and the left about how industries are more concentrated. The Council of Economic Advisers came out with a report recently on economic data with respect to that. Two-thirds of Americans now believe that the economy is rigged in favor of vested interests. There's less opportunity for entrance. There's greater concerns of the widening wealth inequality. So antitrust policy has a very important role in our obtaining the benefits of a data-driven economy while mitigating its risk. But the most important thing right now is for the next administration is really intellectual leadership to understand what are these benefits, what are these risks, and how can we refine our antitrust tools in order to address these concerns. Thank you. Thanks for that and thanks for everyone for coming. I'm not an antitrust guy. I'm a technologist and I'm primarily focused on privacy and security and data generally. And I have a very naive understanding of antitrust law and monopolies. But I understand them somewhat to refer to the exclusive possession or control of the supply or trade in a commodity or service. And then this is kind of one of the challenges having previously been at the FTC and really observing how we're applying antitrust thinking to these new markets. I think it's particularly difficult. So the marketplace we're talking about, the digital or online marketplace often is not exclusively but most commonly fueled by advertising models. And it's a two-sided market, right? You have advertisers on one side, you have consumers on other side, and you have firms mediating either access to consumers' eyeballs or their data or helping sell advertisement to those consumers. And these markets also demonstrate huge network effects where, as we just heard, the value of the good goes up as more people use the system, both creating opportunities on both sides for natural monopolies to the firm's advertising end to having exclusive dominance or access to individuals or consumers. And under the traditional definition, it's hard to say that one firm has a monopoly on the data connect, collect, right? So about consumers' behavior or about even firms' behavior, particularly because the information or data in this case is non-rivalrous, right? So my having a copy of the data doesn't prevent you from also having a copy of the information, right? And my use of the data doesn't necessarily preclude you from you using that resource. And so when you visit a news website, so New York Times or a bookseller, that firm does get the information about your visit about individuals' behaviors, but so do any third parties that they contract with to help them provide additional analytics or advertising services, right? So both those entities get that information. And this is mimicked also in the mobile space or the IoT space where, as we heard, there are app platforms and there are even cellular carriers and there are the apps themselves. And often all of those parties may have access to information about people's whereabouts or locations, their behaviors, and be able to try to monetize them. So traditional antitrust analysis, I think, has a hard time identifying not only what is the resource, but also kind of not only what is the market, but also what is the resource. And the concern for me arises that certain networks have kind of much higher access or certain firms have much higher access to the amount of information, the volume of information, the variety of information than other firms, right? My research, this is quite old now from like 2009, identified certain key firms as having 80% of the kind of market of users, right? 80% access to, for example, across the top 400,000 websites, certain firms were prominent across all of those websites or 80% of those websites where others were not. And only one or two firms have this vantage point. They provide analytics services, they provide advertising services, etc. And I think of that particular fact, and this research has been followed up by others at Princeton and the Web Measurement Laboratory. And it's that exclusive access to that variety and volume of information that makes that firm's dominant position unique, right? It's not that the information itself is non-rivalous or that is limited, it's that only certain firms have access to that kind of swath of information. And it's particularly important to realize that not all data is also created equally, right? So freshness, uniqueness, sample size is critical for monetizing this information. I might have information about your behaviors from yesterday indicating that you might want some ice cream, but that information will get stale very quickly after you've consumed that good, right? And so it's key to have access, a constant input of this information and access to certain populations that other firms may not have access to. And many of you heard the last point described another way, the point about scale, which is the term of art of big data. And while I think a lot of firms have no idea what they mean when they say big data, I think most of us may not either. It is important to recognize that at the scale that actually matters for true transformational insights, only few firms have the ability to reach that scale and kind of leverage big data in that way. And that's because often firms are able to bring data, these firms are able to bring data together from a multitude of sources, right? It's not just one metric or one platform, right? If you ask yourself, for example, how many firms could bring together 80% of the website kind of activity across the web, along with 60% of activity across mobile platforms in addition to a variety of other things like self-driving cars, you know, sensors in the home, smartcams. There's only a number of firms that can have that vantage point and can actually utilize that information. And I think this is the critical piece, right? And you also have may have heard, you know, people make the argument that, well, consumers are free to leave anytime. They're not locked into these marketplaces. And I think this is another critical piece to understand, because in fact, with privacy and with information, consumers are the ones that bear, for example, the harm or the cost of moving across firms, right? So if I give one firm my information and then I leave, then I give another firm my information, I've now doubled my exposure with regards to firms that have my information that can misuse it or breach it or sell that information, right? So the actual choice facilitated by consumers is actually kind of a non-choice given that they're effectively bearing the cost. And the final quick point I want to make, and I think what's the critical point of how we try to address this is that, and it was made earlier, is that we don't really know how to value information in these free markets, in these privacy markets, right? And many might be aware of the settlement yesterday of my former agency with Volkswagen for $10 billion, settling some 500,000 consumers were kind of misled about the fuel efficiency of their cars, right? And I kind of want to highlight that number in contrast with another case from last week where the agency settled for $4 million with a mobile advertising network that tracked hundreds of millions of consumers location without permission, right? And so this is kind of on scale of $10 billion versus $4 million, and this is a smaller firm, but just the number of individuals affected and the sensitivity of information, location information. Another, similarly, a case that I worked on back a long time ago was Google was fined $22.5 million for, as a second strike, they were already under a consent order and they were fined $22.5 million for misleading consumers about their ability to prevent tracking. Specifically, Google used a method to circumvent privacy protections that consumers had in the Safari browser and continued to track millions of individuals for several months in 2011 and 2012. And in contrast, I just, I kind of want to show that as an example of how we value these things in contrast to, say, the Volkswagen order or in contrast to the fact that Google made $36.5 billion in 2011. And so how we try to kind of think about the value of that information and the value the firms are able to realize, as well as how we kind of think about the value of information with regards to antitrust law, I think is going to be critical in how we solve this. Thanks. Thank you. Lisa. Hello, everybody. Thank you very much to the Capital Form in New America, especially to Barry for inviting me here today. It is such a relief to be out of the UK. I cannot tell you the gloom that's going on these days. It's quite horrible. So thank you very much for this opportunity to get away. I would also say that I have prepared a number of slides that you should all have seen. I had prepared for 20 minutes spiel that has now been cut down to seven slash five minutes. So I'll try to make it short. Barry asked me to address antitrust and the intersection with privacy in the EU for US audience. And I'm in a panel on discrimination. So I will briefly talk about discrimination. So discrimination is relevant when you consider big data, as we have already heard, as it can be used to discriminate consumer groups. And in the EU, that may be considered an abuse of dominant positions. So the relevant framework here is Article 102. I had the pleasure and fortune to spend a couple of days in the UK during the crisis with FTC Commissioner Max Rini, who told me that the way they use discrimination here in the US is very much to define the market, how you define the market in monopolization cases. But if I go back to antitrust and privacy in Europe for quite a long time, we had two parallel discussions going on. The privacy camp would discuss privacy. The competition and antitrust camp would discuss that. But recently, those two camps have come together. And privacy is very much a thing in Europe that's considered under data protection rather than competition law. But that doesn't mean that competition law is irrelevant and they can't do anything in this area. It's just to say that the main aim of antitrust in Europe is not data protection. But, of course, in antitrust cases, you can give way to data protection and privacy issues. And you will see I've mentioned a case in the slide, the SNF case. And that shows that there was an information exchange of private nature going on between competitors. And that didn't mean that the European Commission did not take that case forward. So it is still considered something under competition, despite it being of personal nature. So then the question is whether regulators can deal with industries where big data is the core of the product market. And the answer to that is yes. So enforcement agencies in Europe, at the very least, have dealt with data for decades. What has changed recently is the four Vs that Maurice mentioned is that volume, variety, velocity and value of data has increased enormously over the last couple of years. When we talk about data, data is many things. Data can be an input. It can be an asset. It can be a commodity. And it can also act as a barrier to entry. And I think it's just very important to understand that data can be used differently in different markets. And there is no one size fits all when it comes to data. And one of the debates in Europe, one, not the one, but one of the debates in Europe is whether the digital market and online platforms where big data includes personal data, whether that requires special regulation, or whether the current analytical framework we have in antitrust can be used for that. My personal view is that we don't need special regulation, but we can deal that the antitrust framework can easily deal with this. That being said, and this is a big if, that is if the regulators are willing to enforce. So I don't think we need to change the framework. I think we need to change the way we enforce this market. We can't predict the future. What we know is that we can say that consumers still want whether we're in the offline market, whether we're in the online market. Consumers still want lower prices, better quality and choice. That hasn't changed between the offline or the old economy to the new economy, online market. So in the EU, we have had a sector inquiry in the e-commerce market. And the aim of that inquiry was very much to find out whether practices from the offline market or the old economy also exist in the digital market. And the answer to that is yes. So going back to the analytical framework, the most obvious provision to use is abuse of a dominant position. And Maurice, he was quite right to say we have the so-called SNP test with a small significant non-intuitory increase in price. And why we can't really measure price in this mile, we can measure quality. So the SNP test goes from a SNP test to a SNP with a Q in the end instead of a P, really. Because when you look at this, what is so very important is quality and innovation. And then after we have defined the relevant product market, we need to look at dominance. And what has been mentioned before today many times again and again is scale of data and network effects. So if you have an industry where there are scale of data and network effects, it can be presumed that these companies that sits on scale and network effects that they have a dominant position. And then the question is what kind of abuse are we talking about? What kind of contract are we talking about? Typically in these online markets in the digital economy, it is refused to supply, it's tying and it's discrimination. So therefore, I think it's enormously important that enforcement agencies around the world that they focus on market entry and that they make sure that big data does not act as a barrier to entry. And if there is no price effect, because price can be very different to measure on the free part of the market, then they need to look to innovation and quality. And of course, everybody knows that the main argument of these companies is that against one of the arguments, again, intervention is of course that this is Schumpeterian competition. We don't compete within the market. We compete for the market. And therefore, there is a very natural kind of turnover of companies all the time in these markets. But that's simply not true. I mean, Facebook has been around for a long time. Google has been around for a long time. Amazon has been around for a long time. Microsoft has been around for a long time. I mean, you get the picture here. There is not such as there is a new social website coming out tomorrow and then the year after. Unlike the FTC here in the US, in Europe, we do not have an agency that can regulate both competition and privacy. So we don't have combined authority. But that's said, as I said in the beginning, the privacy people and the competition people are talking more and more together. And as I mentioned, we have Article 102, abuse of a dominant position. And what the European data protection supervisor came out with a paper that said that repeated breeds of the privacy rule could be considered an abuse of a dominant position under Article 102. So to conclude, unlike in the US, we do do something in Europe. I mean, we do go after Google. We do go after these big tech companies. So I mean, it's not that nothing is happening. And the Bundeskartell Amp with this, the competition authority in Germany and the French competition authority, they have created a joint report on big data. I personally think that was a mistake, not because big data is not crucially important. I just think it's very important for an enforcement agency not to be seen as biased. And they didn't consider whether there was a problem in this market. They just said, there is a problem in this market. This is what we are going to do. And I'm not here to be the advocate for big tech companies. I'm just saying imagine you go before any of these authorities and you are sitting on a big data set. Do you think you're going to get a fair treatment or do you think they've made up their mind in the first place? Anyway, on that very positive note, thank you very much. Well, we certainly covered a wide terrain here. There's just so many interesting things. You know, I think, since Lisa, you got stuck with a shorter presentation than I think your material certainly deserved. I'm going to direct the first question towards you and I think maybe wrapping in something that Ashkan mentioned as well, which is how does, I mean, yes, the argument that you hear from companies is that they're competing not necessarily between each other but for a market. And then your point was, but look at these tech companies, they've been here for a while and they tend to stick. But, you know, we also do see new competitors arise. I mean, Snapchat is being downloaded at an amazing rate by kids today and maybe the next, you know, platform that has, you know, a potentially dominant position. So how do we come in, how does this play with Ashkan's argument that data is not a discreet good but one that you can sort of deliver again and again? So is it possible that we're sort of layering, like, you know, essentially layering companies on top of each other where they're becoming collectors of the same data again and again and all the consumers use Facebook, all the consumers use, or significant portion use Facebook, a significant portion use Google, a significant portion use Amazon and they're just sort of doling out their data to a number of entities. So how does that play into this discussion? If a consumer can sort of give away their data again and again, how does that cut when we talk about their, you know, inability of new entrants to come into the market or the dominant position and, you know, and the harms, I guess, that we see from that? Well, it was quite interesting because Ashkan, he just said that portability, double the exposure. And it's quite interesting because in Europe in 2018, there will be a regulation where it becomes a requirement that every tech company allows portability. And I thought when I read the draft of that regulation that that was a very good thing until I showed up here today and heard, no, no, this is not a good thing. This is just doubly exposure. So I have to take that home and think about that a little bit. But to your question, you're absolutely right. There are new people coming in. But if you are a small search engine and you are competing against Google, how are you going to do that without a big data set? I understand that Uber, they suddenly get in. They had absolutely zero data when they started. And if you ask now, yes, they have created an enormous data set. So yes, there is. And I think it's about finding a new market, creating a new demand that is not already in there. I think in terms of competing for the market, instead of within the market, if in terms of competing against Google, I just think you have lost that battle from the beginning. If you're somebody like Uber who starts up in a total different market, then you can build up your data set from scratch. And newer and newer industry do that all the time. And Ashken, I don't know if you want to build a little bit on what you see as the harms that come from this widely distributed data sets or widely distributed sort of personal information on each of us. Sure. I do think kind of related, one market to watch. So we're all trying to predict the future. Who knows how it's going to turn out. But I think one discrete market to look at is, it's discrete temporarily. It will be much more broad. But initially, autonomous vehicles and later AI generally, right? So you have all of the companies we mentioned, Google, Apple, Uber, Tesla, which most of them mentioned, but I think Elon was evoked. They all are competing for who can build kind of a safe, practical autonomous vehicle, right? And to do this, it requires incredible engineering, incredible programming and development, but heaps of data, just heaps and heaps of data about consumer behavior or driver behavior, road behavior, road conditions, kind of training sets for algorithms to learn on. It's called machine learning. And looking at how these firms that have, for example, incumbent advantages on data, as well as incumbent advantages on data processing as well, right? So if you look at firms like Google, they have historically given away products like, I don't know, people remember Google 411 or Google 511. It was a service which was a 411 service you could call and ask at like directions or they were giving that away for free to train their machine learning algorithms to recognize speech, which they now use to do things like show advertising on YouTube videos automatically or parse out. But the firm's ability to take advantage of data provided to them versus firm's ability to reap or have access to specific unique types of data. And Tesla in this case, for example, since I think 2012, since they've been adding sensors to all their vehicles that collect driving habits for safety reasons, but they have real-time telemetry on all of their cars in the field on how people drive them and they have exclusive access to that piece of information. So one might argue they have a dominance on that piece of information until someone like Uber or something tries to supplant that with mobile apps or Google tries to do with their street view cars. So I think this is an interesting area to watch about the interplay of data and competitive advantage and AI. It's kind of a hotspot for any sci-fi film too. Yeah. So let's build a little bit on this kind of big data machine learning piece and bring in the discrimination aspect a little bit because one thing of course we've seen is that machines are not neutral. You have sort of the garbage in garbage out problem. If you have data sets that have discriminatory inputs, you train machine learning and you can end up with discriminatory outputs. And I think Jeff, and in addition, if you have smaller data sets, which is emblematic of data sets around minority populations, they are going to be less accurate because there is less data to train them on. So when Jeff, in your work, I mean, as you were thinking about kind of these inputs, and I'm going to turn Maurice to you a little bit to talk about sort of how you see this playing out as you talked about with behavioral discrimination. But what were some of the discriminatory outcomes that you saw from machine learning? And then maybe Maurice can come in and talk a little bit about what that might mean in this kind of larger marketplace. So yeah, there's, in our story, we saw that a false positive has a unique sort of harm, right? You're labeled as more risky than you actually are, which could be, you only get tested once, you only get tested for your first arrest. So it literally is a scarlet letter for the rest of your life when you're tested by these risk scores. But I want to sort of shift a little bit this great paper that came out last year by a Boston, I mean, like last week about a Boston University, by a Boston University student who looked at these sort of biases in big data. So Google has put out artificial intelligence learned data set called Word2Vec and based on data that they own. So they own, they trained this natural language processing algorithm against a Google news data set. And what they found is they found underlying biases. So this algorithm learned, it basically can learn analogies. So he, when you do he, doctor, and then ask it what the noun is that corresponds to she, you get nurse, right? So even in big data sets like this, you have those hidden underlying biases that are very hard to correct for that's that, that algorithm is now widely used for things like autocomplete, you know, it's used for understanding semantic meaning and search results. But it has these underlying biases because it was trained on, unfortunately, I'm a journalist who was trained on the news. And I guess the news has some sort of gender things going on in there. But no one's really looking, no one's really asking, right? If we if we take all of these data silos that are unique to a company and throw them at an algorithm, and it spits out something that looks, you know, accurate or right, we don't ask whether or not there are these biases or in which way that it fails. And that's, I don't think, by the way, it's just the media that has biases. It's a little more endemic than that. And then we go to questions or are we done? Oh, OK. Maurice? Yeah, a couple of things. I mean, one of the things that we asked was, is perfect price discrimination available in our day and age? And that would be the ability of an algorithm to identify for each of you how much you're able to pay. And we spoke with several people in the IT community and they said, no, it's not possible within the next five to 10 years. Just given there are so many different problems, including the ability. So what's going to happen is that you're never going to be able to identify how much you are able to pay, but they're going to put you in a group. And in that group, they'll have some idea about what's the maximum you're able to pay, like being able to discriminate. And Apple, for example, with your latest iPhone, they promise that they won't segment you in any group smaller than 500 people. But these will be other people who have similar purchasing behavior like you, similar habits and the like. So one of the problems with behavioral discrimination is what if you're put in the wrong group? And now you're in a group that might be deemed a bad credit risk or you might be susceptible to some item. So that's one problem. The second problem is the super platform in being able to discriminate has a lot of power. And is it just going to be on consumer products, but it's also going to be on the news feed. Most people get their news like younger people get their news through Facebook, for example, and some of the other super platforms. And now it's like, what sort of stories are you going to get based on what the algorithm believes you would be interested in? And how could they manipulate your behavior, let's say before the elections? There's some research with respect to that, to the extent that the super platforms have power to influence elections. So you're going to have a couple of things going on. One is discrimination based on the category that the company believes you would be best in. The second is going beyond products, elections, and news, discrimination in the news that you see is going to be different from what the news that your colleague might say. That's great. So I too lied. I thought we had a little more time and we would get a chance to do Q&A, but I'm sorry that was on me. I didn't mean to. What I would like to do though, really quick, because it's been a panel that we just scratched the surface on, is it would be, I'd love it if our panelists could leave us fairly quickly with just one trend or area that if, as you consider this over the next, you know, 18 months, that you say, boy, I'm going to watch this because I think it's a really, it'll be very revealing for where some of these issues are going to go. If you could, each of us kind of give me just a real quick one, just to give the audience something to kind of take away and keep an eye on going forward. Lisa, do you want to start down at the end? Yes. And this built on my previous comment with his conglomerates. While Uber built up its own dataset, and what they have done is, of course, when they built up that dataset, they expanded from driving around people from town to town into the food market. So suddenly a company that was a taxi company becomes a food company and vice versa. I mean, what happens is they become from a single company to a conglomerate, they expand on one market into the next market into the next market, and that's how they build up their market power. So we haven't really been focusing much on conglomerates. And I think that's where our focus needs to be. Thank you. I'm a little bit cynical. So I think that people are going to start noticing that artificial intelligence and big data isn't as accurate as it purports to be, and we are starting to see that with sort of stories about racial bias and Google search image results and the paper that I mentioned. And I think over the next 18 months, there'll be a lot of soul searching over the fact that a lot of these algorithms, the algorithm that I just mentioned, the Google algorithm, is only 40% accurate, right? So where are the biases in that? Great. Ashka? I'm going to leave with a quick kind of thought experiment that touches on these two points, which is that there are intentional biases. The Volkswagen example manipulating emissions was an intentional bias in the algorithm. There's also unintentional biases. And the real question that I've always struggled with is how can we know or how can we tell? Particularly when they're non-public, when the systems are not auditable in a good way. And the thought example that I always use is, you know, we all now use, you know, mapping software like Bing or Google Maps or to travel around every day, right? And these are now becoming APIs that are the basis in the same way. Search results, all of the other services like Uber and others will use the mapping software. And the question that I often ask is we assume them to be the fastest directions from point A to point B based on traffic patterns and distance, etc. But as these companies are conglomerates, right? So Google, for example, owns a display ad business for like billboard ads and they also have retail partners. Would you have any way to know that the directions you're receiving are actually the fastest or quickest or if they've been kind of slightly manipulated to lead you by a billboard or lead you by a store or lead you by a particular? And I'm not saying it happens. I'm just saying we don't have a way to measure or tell. And I think in these domains, whether it's intentional or unintentional bias, it's quite a precarious place to be. Maurice, you'll get the last word. All right. So where does the power lie? In 1990s, it was Microsoft with their platform. Today, the power lies with the operating system for the two dominant mobile platforms, Apple and Google. So where is it going to go tomorrow? And I think this is what one of the themes that you get from our panel, it's the convergence of artificial intelligence and the internet of things. And what you're going to see is not necessarily new players, but the four dominant super platforms today, Amazon, Google, Apple and Facebook, they're now investing in that. So the thing to be looking out for where the power is going to migrate will be personal assistance. So rather than having a search engine, which makes you do like three iterations, like you have the first search, then find and then call, you're going to have a personal assistant that's going to guide you seemingly to your benefit, but there's a lot of power then in what they bring you to. And then the second would be self-driving cars and not so much as for the technology itself, but really as an advertising driven platform so that you can then be then subject to or benefit from the super platforms, other products as well. So that's what I would say. Look out on the horizon. Great. Well, please join me in thanking our really excellent panel. Thank you very much.