 Hello. Welcome. Is this working? Yeah. Welcome to all of you. Can you please move towards the seating part of the room? We just need to do a little housekeeping and opening remarks before we actually start with the presentations. And as you have seen the schedule is quite tight today and we would like to have time to hear everyone and have maybe a couple of questions after every paper that are immediately directed to the paper and then hopefully we'll have more time for broader discussions later in the day. So that's our way of trying to keep the thing going. I'm Massimo Mazzotti. I am a professor in the history department, but I'm also the director of the Center for Science, Technology, Medicine and Society. And we are organizing this event with the help of many other people and institutions on campus, for example, BIDS, which is giving us this great space for our first day. I don't think I want to say too much, but BIDS, STMS, CSTMS is an STS center. And our affiliates are from all kinds of departments across campus. We have a number of projects and priorities at the moment. And one of them is algorithms. And we started with these kind of conversations maybe five, six years ago when big data was the term. And somehow the choice of going for algorithms was an aspect of the fact that many of us were interested in grappling with the actual technical details rather than with more general notions of impact and implications. So that was, I think, a signal in that direction, which is still what brings us together on the humanities and social sciences side of campus now when we talk about these kind of things, when we talk about data science, for example. So I hope this will be the beginning of a long series of events. And I hope that Berkeley and CSTMS, BIDS, other places here on this campus might become really the home for a long-term conversation. There are many things that are going on at the time in terms of how data science is going to reshape the way we do research, the way we teach, the way we understand interaction between different parts of campus. This is not only happening at Berkeley, of course. It's happening in many other places. But I think there are some distinctive traits of what's going on on this campus that makes it a good place for these kind of conversations. One of them is a very strong presence of the social sciences together with leading departments in computer science and engineering and so on. And that makes for a fairly balanced and interesting conversation in between those parts of campus. And also the fact that whatever is going to happen in the near future in terms of data science and the institutional configuration that it will take which is still the object of a very intriguing conversation that is going on right now on this campus. But something big is going to happen at some point in terms of transformation of the way in which we do research and the geography of the research geography of this campus. But what is clear is that social science is going to be an aspect of that. I was just discussing about a new data science major that is going to start next year. And everyone agreed that social science should be an important aspect of that major. That there should be a path in that major for students interested in the social implications of what they're doing. So this is just to say I think this is a particularly congenial environment for conversations of this kind. And so that's the background just for those of you who are not from here giving you some background of where are we coming from with the idea for this conference. And also the fact that I hope this is just the beginning of something. Not the conclusion. So I leave the microphone to Morgan who's the really amazing driving force behind this conference who will give you some more specific details of what's going to happen today and tomorrow. Thanks Massimo. All right. Thank you all for coming. 2016 has been a really interesting year for algorithms to say the least. We have the year starting out with GO. Seeding to an algorithm much like Chasted back in the 1990s. We had a space shuttle landing that was really algorithmically driven. Very perfect landing of a SpaceX. Of course there was a later SpaceX explosion as well. We had the first self-driving truck do a delivery of beer. Algorithmically driven. We've had really taking off of the gig economy. This isn't new to 2016 but I think it's really grown massively in this last year which we'll hear about in some of our papers today. And of course there's been a lot of talk about algorithms in news and the role that algorithms play in segmenting us and dividing us or perhaps bridging those divides. For example Facebook firing all of their editors and relying on algorithms for those trending topics is something that came up this year. So I think we have a lot to unpack over the next couple of days and I really look forward to seeing everybody's paper. We have really a phenomenal group of people here. So I hope all of you who are attending can attend as much as possible. Just a couple of little housekeeping things. One is that the bathrooms are a little bit hard to find. I'm actually going to put Stuart on the spot if he is back there to describe where they are. He'll come up in a second. Also those who are presenting today we do have the capability of live streaming today. So if you are interested in having your talk live streamed or more specifically if you would specifically not like to have your talk live streamed and you are presenting today, please let one of us know and we'll make sure to accommodate that. Also this is a working space. You can see it's a great venue for conferences but there are also a lot of spaces around where especially later in the day people will be working. So obviously we'll still present and we'll still make some noise but we should also respect that. All right Stuart, if you could describe. The most important thing to learn to know. The bathrooms, yes. So if you go outside and then go around and then they are by the exit. So if you would imagine, please don't you go out instead of out of those doors while people are talking. These are also sort of locked from the outsides and you get back in. But if you imagine if you went out and just took a far right they are outside the metal detectors by the far exit. So sorry they couldn't be more convenient but so you'll go out and around and back and then around here. All right thank you very much. All right so I think we'll start getting set up. I'll hand the mic back over to Massimo who will be our session chair for the first session. Okay so we start with the first speaker who's Emily Paul from our own University of California Berkeley losing and reclaiming the body in predictive criminal justice. Okay. Hi. All right. So I'll just jump right in. I think many of you here are aware predictive risk assessment algorithms are increasingly being used to inform decisions in the criminal justice system across the U.S. and decisions about detention. So whether that's pretrial detention, sentencing after someone's been convicted or decisions about parole and probation. And some form of these risk assessment tools is in use in 48 states at some part of this process. So broadly speaking these tools rely on a model based on historical data to assess factors about individuals that are predictive of recidivism and assign a probability of recidivism. And so I look at Kristen Rosenblatt and Boyd's definition of algorithm and automatic rule that uses numerical inputs to produce some result. And various approaches to calculating recidivism risk have been around for a long time. This isn't new to big data or the current focus on predictive algorithms. Actuarial risk assessments have been in use for about a century. But as the focus on kind of quantification and predictive algorithms increases these tools are being used more broadly and are also being used for more decisions. So more types of decisions within criminal justice. And two tools that I look at that are too widely used are the level of services and the compass tool which was the subject of that recent pro-publica investigation in Florida. And these tools were both initially developed to help target risk factors. So to design rehabilitation and focus on things that contribute to risk. But they're both being expanded in what they're being used including most recently in sentencing decisions. And so this is just looking a little bit at this is from the LSIR, a sample. So you can see some of the factors that are being used like have you had three or more address changes in the last year? Do you live in a high crime area? So this pretty quickly already shows some of the concerns people raise around these factors being very correlated with race and income and really perpetuating biases that have long existed in the system. And the answers to these questions are used to categorize the person into give a risk score and categorize a risk level. So in this example they have a 28, they're in the high medium group and that is associated with a 44% probability of recidivism. So this would be a report that like during sentencing or a probation officer would use something like that. So there's been a lot of discussion about this and concerns raised about how these systems reflect and exacerbate discriminatory bias that has long existed in our criminal justice system particularly against people of color and low income people. And people have talked about different means, legal and technical means of trying to mitigate some of this bias. And former Attorney General Eric Holder spoke about this particularly in the expansion of these tools to sentencing. But what I want to focus on today, my argument today is that even if we're able to, even in a perfect thing if we're able to eliminate this bias or at least mitigate it that there's still some fundamental assumptions that go into using a probabilistic prediction to make detention decisions, decisions about actually incarcerating someone that we can't really get around and that we should surface and negotiate with more in deciding whether to use these systems and in how, you know, training people and how people actually use them if they are implemented. So, um... And... Okay, yeah, I'll move on. Sorry. So the first is representing people as what Gavin Smith calls data proxies. So, you know, coding and highlighting, selecting attributes and representing those attributes in data. And this is, of course, not limited to criminal justice context. This happens anytime you're creating data representations of people or phenomena. But, um... And it takes an embodied individual, this person, who's embedded in a very rich social context and selects certain attributes and ways to code those attributes so, like, three or more houses in the last year, yes or no. And I look at what Hales calls the platonic backhand. So going from this very complex material reality to an abstract representation, a simplification. And, as she says, this is something that has gone on for centuries. This is how we come up with theories and find patterns. So this isn't necessarily inherently problematic, but we need to remember that we have done this when we then use those representations to make decisions. And so that gets to the second move of taking this abstracted model, applying it back to an embodied individual, which is what Hales calls the platonic forehand. So going from this abstraction to seeing the world as actually a fuzzing up of that kind of more pure form of the model. And with criminal justice decisions, it's not just interpreting someone through this. It's actually making a decision about, you know, their body, about keeping them behind bars. So I feel like it's kind of the platonic forehand plus even a little more extension of that. And in doing this, that aggregate, frequency-based prediction of the population and the historical data is mapped directly to a prediction of a one-time event for one person. And this is something that Ian Hacking talks about in the history of probability. There's dual concept of probability. There's frequency-based and there's degree of belief. And these get kind of conflated from a frequency across a population to will this one person in the one life that they have coming forward, will they commit a crime? And this happens all the time. It's not, again, unique to this context. And there's a very popular or, you know, good example of this right now around the election. So Nate Silver predicted about a 72% chance that Hillary Clinton would win the election. And there's been a lot of discussion about how these predictions could be so wrong. But they're not really wrong in that the way that those predictions are generated is that daily 538 would run 20,000 simulations across their models of the election to see what are all the different outcomes. And from that frequency say, okay, in most of these simulations, in 72% Hillary Clinton wins, so that's our prediction that Hillary Clinton will win. But what we don't know is in the one time that it actually happens, will it fall in that 72% or will it fall in that, I can't do that math clearly, 28%. So now I just want to walk through the process of constructing and using a predictive model and criminal justice and look at how these assumptions kind of, what kind of limitations come from these or what we should be considering. So broadly four steps involved. You have to collect or get access to historical data where you know the target outcome, you know if people are sedivated. You have to construct a model, aggregate that data and construct a model and then generate a prediction. So use that model and take data from a person where you don't know the target outcome and predict that outcome and then use that prediction to form a decision about whether to detain that person, either pretrial or for how long to sentence them, where to sentence them and so on. So I'm going to walk through the level of services inventory tool. So Andrews and Bonta who created this tool started with a data set of over 23,000 people in prison on probation or on parole. And so these people are represented in this data set as data proxies. There is a process of coding and highlighting that has already occurred to represent certain factors about them. And this always emphasizes some things and leaves other things out. In criminal justice, this coding and highlighting is especially not straightforward because all the instances of the target outcome of criminal behavior aren't necessarily captured, right? So we know that policing happens very, is not uniform across different communities, different populations. So who even gets noted as having committed a crime? That's not all being captured. And the definition of what constitutes criminal behavior can be contested. So especially in this case, there is a lot that is not straightforward about this coding and highlighting and creating this data set in the first place. And then aggregating the data to construct a model. So Andrews and Bonta look for correlations between factors that criminal theory that they hypothesize would have some correlation with recidivism. And then looked in the data set to see how frequently that correlated or how well that correlated with someone committing a crime on release. So they need to first aggregate the people in the database. So treat everyone, anyone who has three or more address changes in a year, yes, that's the same. And obviously there's a lot going on behind that yes there that is varied across people. So it's ignoring that context of that embodied individual that's not being captured. And we don't have access to it later in the process. And Andrews and Bonta say that their focus is on the criminal behavior of individuals rather than on systems and on sociology. So they're also acknowledging or saying this is what they're trying to do. And then generating a prediction. So collect those same data points from the new person, a defendant who you don't know what their outcome is going to be. And based on their answers to those factors group them, categorize them with the historical data and assign a risk. So again here they get a risk score, a risk categorization, and that's associated with a frequency based probability of recidivism from the historic population. And then actually using that prediction to inform a decision. So and as Kristen Rosenblatt and Boyd talk about in their piece we don't really know, and I think you're going to talk today somewhat about what we do know or what we can talk about a little bit. But we don't really know how these systems are being used by different judges. There's resistance certainly. It's not a simple straightforward story that someone just gets this probability and then they make a decision. But in the ProPublica investigation they did speak to a judge who talked about getting a high compass risks for a defendant which led them to overturn a plea deal. And then on appeal the defense actually brought the creator of the compass algorithm to testify and he said this wasn't intended to be the single source for a sentencing decision and the judge actually reduced the sentence back to what he had initially been considering before seeing the risk score. So there's some different examples where we see this having an effect and also there's research in behavioral economics and psychology that talks about how people seeing these kinds of scores influence people even if they're not necessarily making a direct decision based on them. So from here I guess for me it just seems to call for more understanding of how people use these tools and coming up with ways of thinking about and keeping the assumptions behind them more visible and available to people so that when decisions are being made about whether to implement these at all or when people are being trained to use them those are more available to people and people actually are negotiating with them and considering them and how they're using the tools. That's great. So questions? Thank you for a great talk. One thing I wanted to point out from the technical side is that in this articulation of the different steps of making a predictive model that's really important is evaluating how accurate that model is and it seems like this has really strong implications for how judges would be using these predicted recidivism rates and actually making decisions because one thing that you would want to know is that if you have a score that you associate with the 44% probability of recidivism happening is that probability actually calibrated? I mean is it true that in the cases where you have that 44% of a chance that 44 out of 100 people actually do recidivate? Yeah. It seems like these are really important questions to address or in thinking about too as you think about how they're actually being used in the context of making decisions. Yeah, thank you. So and yeah, Andrews and Bonta the LS tools they do there are some validity studies but one thing that I've kind of been thinking through is even if the validation is out of 100 people who get this risk score do 44% actually recidivate it still doesn't address the mapping that probability to one individual person because you could have a perfectly accurate model that 44% of people do actually recidivate but what you don't know is which is this person in that 44% group or not. So that's something I guess also that seems like another layer of that. Yeah. Hi, thank you so much. That was a great talk. So I have a question about in thinking about culture the spaces this is happening in because you have on the one hand the courtroom which to me invokes questions of expertise and how number is used in judging and then you also have the prison system in thinking about data it can kind of create a closed system and then there's questions of imprisonment things like that so that's a big question but could you elaborate on the spaces behind the data the context? Yeah well I think also the next talk is going to be pretty specifically or one of the next ones about expertise and how people actually respond and so I mean I don't know that I have that much smart to say about that I think I would like to understand it more and ideally like in those contexts to really see how people are responding to the system. Yeah. Thank you. As a lot of people note the criminal justice system is no longer or never was just in or out of it and that everybody, almost everybody involved in it is in some kind of weird middle stage between being in prison or out you know parole halfway house what have you do these tools that people are using that judges are using or that the analysts are using do they take account for the kind of gray scale in between being in prison and being out? So do you mean when they're assessing like the factors that they're looking at do they account for that? Yeah, like you know does recidivism always mean when they're calculating this does that always mean returning to prison or does it mean like are they trying to predict also you know increasing the period of your parole or all these weird like mid spaces or like losing your license or something like that. Yeah, that's a good question. I actually don't have the answer to how they define recidivism. I would think that it would be like convictions because even though that's problematic to like arrest is really problematic convicted and then don't go back to jail you know and I don't know if that counts as recidivism. I think yeah I think convicted would regardless of what happens because of that conviction um yeah, does that answer your question? Emily that was a great talk and I apologize for arriving late and that pertinent I think because I saw that extraordinary category in there which was could make better use of time a ripple runs around the room I mean how are those things assessed? I mean if we assume from that we then get this concrete data and we have 40-40% recidivism but how do we make even a 44% assessment of could make better use of time? Are there given instructions etc? Yeah, I mean people are definitely trained to administer this instrument which like you know which has been created based on looking at this historical data but of course there's still the individual probation officer or whoever is doing the assessment um yeah and that feels like a whole another I mean that's a super important part of this that merits a lot of exploration too like how do people just code these things often that are like a yes-no um for something that is very subjective and complex um yeah that was a partial answer but they are through interviews yeah so they're being assessed in that way okay I have a quick question I was curious uh at the end when you say you know what to do what is to do now you mention a need of a better understanding of the mechanisms of decision making algorithmic decision making process is that a straightforward process in this case? You know issues of transparency I mean is that something that can easily achieved? Well I guess there's a whole conversation about whether people understand all the underlying correlations and why has this been considered predictive um I'm actually with that I was more focused on just I find this thing about the probability itself just so compelling like you know if you see how do people interpret probability it's really abstract and then they're using it to make this decision that has really like concrete consequences so even there's a whole conversation about whether they would understand or was arrived at in terms of what are the factors and why is that factor predictive which is really important too but with this I'm even just thinking like how do we interpret probabilities we and can we kind of remind people what this probability actually is this is a frequency across an aggregated population um so even specifically on that even though there's definitely work to be done on understanding it in other ways and pushing back against it in other ways but I'm maybe asking a technical question due to my lack of understanding of like the nuts and bolts but I was struck by this this idea of um you know you have these features like how many times has the person moved within the last such and such a period and let's say you're administering this in the case of a particular individual and you know and you learn that the person has moved more than three times or whatever but it so happens you know the background information that allows you to see that this isn't really the type of you know there's something exceptional about this case um which isn't what you might imagine to be the normal case of someone who's unreliable, doesn't pay their rent, messes if you want to give an explanation so in a case like that when you're administering when one is administering um in order to make some kind of decision what happens then does that just kick you out completely of the model and say oh the whole model doesn't apply or do you say I have to blind myself to this external knowledge that I happen to have because for the most part this model is going to give reliable prediction and now that I've bought into it I must just go with it I mean I wish that I you know had spent time with people and actually seen how this plays out because I would imagine that it's different for different people and I think the three or more addresses is a little different like some of them leave a lot of room for interpretation like could make better use of time you don't you there's not a forcing you into um just tallying up okay do they do X, Y and Z whereas the housing one is a little more it seems more straightforward um and maybe you can speak to this too uh but I think that there are cases where people do kind of give a different answer because it is this qualitative assessment based on an interview um and they do kind of accommodate or adjust it to reflect what they think the risk score should be even um so they know hey this person really is medium risk so I'm gonna kind of make that come out that way um yeah I don't know that kind of do you want to say something more about that in your one last question otherwise we might actually use uh actually thank you so much for your talk and we can use a little bit while uh the next speaker set up the computer there are many seats empty seats up here on on the front so if people want to move from the back this is a good time to do that right yeah then okay um so the next speaker is Kyle Kubler department of communication university of Washington uh state of urgency the implications of protocol procedure and data visualization in police software okay um thanks everybody uh excited to be here um I grew up in the Bay Area so I've like been walking around this campus like my whole life and it's cool to be here doing something so um yeah it's fun um yeah so I'm gonna be talking about the state of emergency in France that's been going on for the past year and sort of talk about the um the software that the police use and have been using for a while um and how that use sort of changes during the state of emergency um and how government procedure shifts and kind of changes the implications for what happens um how we arrest people uh and and kind of like how algorithms play a role into that so uh it's called urgency too um in French the word for state of emergency is l'état d'urgence and um so there's like a sense of urgency or speed uh that is kind of implicated into the word in French and I think that's kind of helpful or interesting to think about as we kind of continue to think about what that means so uh I sort of started off my uh research with some couple questions so the first one was how do algorithms change the possibilities of policing when government procedures altered so that altering is the state of emergency um and this question is sort of like on a more theoretical basis is kind of founded in the work that um like Louisa Moore have done on um sort of algorithmic state of exception and kind of pulling from a goblin so this is like a little theory heavy I guess um and then the next question is what does the state of emergency tell us about how societies of control are created so again drawing in that question Moore some from Foucault talking about uh sort of societies of control uh and soft power I think there is a notion that uh whether you know we agree with it or we don't that uh soft power or surveillance can kind of uh present a state that is maybe less violent um or more predictive in the sense that it doesn't have to be uh it doesn't have to use hard power it doesn't have to constantly discipline its citizens and that the state can sort of rely on surveillance to deter crime so that's kind of one of the uh one of the assumptions in that question that we'll try and look at through this presentation um so I'm going to give some background about the state of emergency just sort of how it's developed over the past year um and then I'll talk a little bit about the software that's being used and then some of the implications later so um to start off the state of emergency has been going on since the terrorist attacks in November uh so it's been about a year now um it was originally um started for just about two weeks and then it has been kind of re-ratified multiple times now and it'll be um on until the end of the year um and the the biggest sort of aspect of um government procedure that has changed is that um the police generally do not need um a prior prior judicial approval to do any of the following so house raids arrests, seizure of property um data is a big one of that property um shutting down websites preventing public gathering um so there's sort of no more need for um prior judicial consent to go ahead and do all of those things what's resulted from that is over 4,000 house and business raids um and 500 house arrests um that have happened over the past year it's been 18 terabytes of data collected um so we don't actually know exactly the composition of that data so um if that is like 18 terabytes of everyone's like bootleg copy of Game of Thrones then maybe that's like not a lot but um if that's emails and contacts um that's quite a bit of information so um something to consider there too um there have been lots of arrests um but about only one prosecution for a terrorism related case actually um with six uh investigations so a lot of what the police are doing here the other arrests that are not terrorism related is that they the police sort of have information mostly about um like drug smugglers and things like that uh people that have drugs that they don't have enough evidence to make a case for um they use the state of emergency to go and raid their house take the drugs and then make a case for it afterward um the state of emergency has also been used to put like a lot of pressure on activists in the the COP 21 sort of environmental conference that happened last year um as well as sort of persecuting people that were out on the street for um the the labor laws that have been passed recently so that was sort of an ongoing struggle in France um over the summer so the software that they use is um IBM's i2 analyst notebook um so sort of created in the kind of like early mid-2000s it's been updated quite a few times um there are about three components of this software uh first one is data mining uh GIS and data visualization so I'm interested about uh looking at data visualization just because it is so integral to a lot of the data mining algorithms that are used um particularly when it comes to policing um and uh i2 analyst notebook is not predictive policing uh so some of the police software that are probably the most well known are kind of in the news now are Predpole and Pound here um these use machine learning algorithms to sort of make predictions about where crimes are gonna happen and how to best allocate police resources into certain cases um this is not that so uh there is like a pretty large involvement of human analysts in taking the information that's gathered from the data mining algorithm and then kind of creating um the visualization so if you kind of just upload a data set into analyst notebook and you have it on the sort of link and node graph of the social network analysis that we kind of see up there uh it kind of is just like all scattered everywhere and it's like part of the um the person who's working on it the human analyst to start and figure out what goes where um as you can see it can kind of be mapped onto um actual maps uh sort of collect data points um and these data points are sort of conversed generally as far as the policing aspect goes um we'll comprise of conversations between people um the use of different objects so like so and so drove this car to this location at this time and talk to that person um and then this is how the police sort of use the information that they gather um from multiple databases to create a criminal narrative of someone um and you put them in a network amongst people and then kind of decide how important um someone in the network is and who are we going to arrest and sort of like who do we who do we take out that's going to be the most valuable to like breaking up a criminal network um another reason they uh one of the reasons that this isn't predictive or um is that it's kind of difficult to use the machine learning algorithms like Predpolar Palantir when you're dealing so those types of algorithms are generally pretty good for things like um is there going to be like a car burglary or is someone's house going to get broken into because those are uh crimes that you generally have like a lot of information about and you can say like okay we can make like sort of a statistical guess here to say that this is going to happen at this time uh but since terrorism happens like uh incredibly infrequently in like the sort of broad spectrum of crime that uh you can't it's not as easy to make those kind of judgments so the human analyst when it comes to terrorism is still very important so um sort of more technically speaking the the algorithm that is a list plus algorithm um as sort of as flair johns talks about um so it's good at taking information from lists um and to different databases and compiling them into visualizations um it's good at providing human analysts with tools so it has social network analysis things like that um but it's it's not very good at creating coherent visualizations like I talked about so that's sort of why it's not predictive and why people need to be involved there um but some of the goals for this that I think kind of underlie how algorithms are used in policing which is kind of one of the critical areas to think about um is that it drastically reduces labor uh it increases police processing speed um and and gives them data driven results so we sort of ask questions about what to do uh when it comes in policing is that the sort of the the aspect of labor I think is incredibly important because um it it does it makes things it makes the processing speed so much faster and the ability to collect a lot of information and then put it through algorithms to give some sort of coherent result uh is something that wouldn't be able to be done at the speed um that it is without them or if there was like you know basically calls to I think like reduce the amount of influence that algorithms have um are kind of like directly admit from administration by saying like well then we need more money because we need to hire more people so I think that's kind of like a question that's always at the back of that um of that debate um so this is sort of yeah how I2NL Snowfoot kind of plays into that um so again visualizations are are pretty key uh in working with these algorithms and understanding them from the policing standpoint particularly in France um because they help create cognitive shortcuts uh and they allow so this is sort of yeah a visualization of I2NL Snowfoot here um but so they allowed us to make sort of our best informed decision um and they will kind of like you know this is this is an example of um picking someone who is like the most important in that social network uh so this is kind of the the software saying this is the person you need to arrest um so some of the the biggest questions here that I sort of have um are that uh algorithms and this policing software are like really the central aspects of creating data driven type of policing um and I think that the change in government procedure that we see um with the state of emergency uh sort of shows us the capabilities of what algorithms can do without sort of the kind of like the legal constraints of your sort of standard western democracy um and it's uh I think it helps like see some of the broader questions about like human rights and civil liberties and things like that that kind of go into play here um so sort of my my hunch here about what's going on uh is that uh the state of emergency in France shows a police force that is sort of more interested in gathering people's data than it is about clearly making arrests on people um so we have 4,000 house raids with one formal terrorism arrest uh and then six sort of six investigations um statistically that's like not that good and again like terrorism doesn't happen that often but still like pretty bad um and I think what it shows especially with the 18 terabytes of data that's been collected um is that like the data is actually what's most interesting um and that if you can find certain instances like the state of emergency where you don't have to get judicial approval before going in to raid someone's house or someone's business you can take their cell phone you take their computer you take their external hard drive whatever it is and then just kind of deal with the data later right because the the algorithms the sort of list plus algorithms and these data visualizations that are implicit in I2NL's notebook and many other software they generally again up to a point generally will work better with more information um so the more data that you have the more sort of coherent social network you can create and then the better your algorithm is going to work theoretically in terms of figuring out who's the right person to arrest or who's not so the state of emergency allows the police to go in and say you know we don't think that you're going to be implicated in anything like we don't really but you might you know your cell phone might have some information about someone or your hard drive you know maybe you talk to people or maybe you go to this mosque right like mosques have been shut down in France all the time um and they take that information and then they can kind of create the kind of like identity of the terrorist or of the potential suspect after they go through and go through with these house raids and put people under house arrest um so there was the National Assembly had sort of a uh investigation of how the state of emergency was going what were the procedures of the police and this was sort of an interesting quote that I pulled out um so one of the colonels was sort of talking about um what's interesting to them about the um about house raids or sort of what's important to them and he says the best part is the data because sometimes we can find connections to logistic networks uh nobody sleeps with their AK which I thought was which is like a kind of like a funny joke there but it's also I think true is that um it becomes under certain circumstances it becomes difficult to get judicial approval beforehand to prove that someone's going to do something um because criminals are like criminals can be fairly smart they might not sleep with their big machine gun under their bed but that cell phone might have something that's incredibly important to unlocking the terrorist network so the state of emergency becomes an ability or becomes a way where in which you can just kind of go in there and grab the information figured out later so to come back to um a couple of the the answers with the question mark because I'm not super certain about them um but uh to to some of the questions questions didn't have a question mark by the way it was like these are certainly questions and these are maybe answers um but uh so I think that um when we're thinking about sort of state of exception um or agamban or some of these like um ontological problems with like western democracy I thought it was kind of interesting to look at how these algorithms act in a state of emergency right this is sort of like agamban state of exception happening right now in France um so I think that right it puts data distraction over arrests um and then secondly what does the state of emergency tell us about societies um I think that maybe one of the things that is kind of missing from some of the um debates about kind of control societies um or maybe like post modernism especially people using Foucault to think about surveillance and soft power um is sort of where the ability for that soft power comes from and I really think that what this shows um is that the sort of the soft power type of surveillance state that is supposed to deter uh criminal activity in more of a non-violent way actually has its origins in like very like violent direct appropriation of data um through these measures of the state of emergency and house raids and so that like it is only after the police come and break down your door um and put you under house arrest and steal your computer that they can say like ah ha like now we have enough information to create these criminal networks that can then sort of like surveil you and stop criminal activity and then like we can be peaceful um so for like fellow Marxists out there this is like the primitive accumulation of data that like has to happen at first in order for there to be like um this kind of like um system that is created so I think that this kind of like and this this type of um activity right this this tactic of just going in there and taking the data and figuring out later is really only possible at the scale that it is right now because of the sort of list plus algorithm and the data mining algorithms that are involved in this so um I think those are sort of two aspects two of my potential answers to those questions that I had coming into this research um but I'll I'll in there thanks a lot thank you so questions for Kyle thanks very much for your talk in your account I was thinking the IE2 EBM software actually could be used for very different purposes I think it could be very useful qualitative analysis for business modeling for social media analytics and of course for Policing too of course these things have they have something in common but I was wondering about the rationality of or the epistemological framework this the that such a system um enable I was wondering about what actually this general purpose tool which one what kind of epistemological framework produce in the user with different visualization or I was wondering if you reflected also on this thanks yeah I mean I think as far as the the type of vision or mindset that comes out of looking through these softwares and interacting with them um and kind of like mapping that for real life is the assumption that like someone is culpable in this network that they're like there is a target and we can arrest them and we can find them because again like I think like like you pointed out which is very true is that the software is not just used for policing I mean this is sort of like a broader um social network analysis tool so you know from a business standpoint you say like oh like there's my there's my segmented um there's like my segmented population these are the people that I can sell my to best um and they do this now in advertising all the time is they take these social networks um they sort of like have these type of visualizations up here and they say like okay if we can if we can target our advertising to this person they can then advertise to all of their other friends and like this can be the pinnacle this gives like this is the person who's most central to that network um so I think as far as what that does for the police analyst or kind of like the state in a broader sense is it looks at these networks and says like someone is um someone is going to be at fault here and like we can find someone in this network and even if they don't have a high percentage chance of being the one that's actually it they go for it right and it's like similar to um Emily's talk just a second ago is like these like we have these percentages and these probabilities that like may or may not way out in the way that we think they do but you it I think it forces a decision of saying yes we have to do that and again like that urgency aspect in the state of emergency I think like only like exacerbates that more that sense of like having to make a decision and like having to just go with what you have and hoping that it's like you know it gives you that data-driven response that makes you feel better about your decision but maybe not necessarily the case. Thanks that was great um I have a related question um I was I was thinking about this slide that came I think before this one maybe one or two slides before where you're talking about um why the police are using uh yeah exactly so and then the list is kind of the goals are yeah it's almost like it comes from the from the products you know owned flyer right it's super tautological yeah reduced labor okay I get that um increase processing speed data-driven results these are these are very generic um tautological kind of pretty things that just sort of become themselves and so I was wondering kind of what your take is on what on what what else is happening or what these things are producing you know processing speed of what data-driven results about what um I think you just answered it a bit with the with your answer to the last question but um if you had to give an answer to this if you had to fill out these three bullet points in a way that wasn't sort of their language what I'm curious what you would say um yeah I mean I actually would probably I mean I think I'm sort of like critical of policing and like state violence kind of already so I think like I think actually using their language is like kind of works for at least me as well in the sense that like um there's been sort of this um like the Vigipurat law in France for um almost like five years now I think and what that's done essentially is that's like increase the amount of policing in France just in general but they haven't necessarily been able to pass um sort of like through the national assembly and things like that they haven't been able to pass laws up until recently that have actually like increased funding for police so police have been working overtime like crazy and they don't have they don't necessarily they can't unlock the the sort of like um the like the citizen support for those laws that were passed so um you just have a police force that's kind of tired and overworked and um these software I mean ITVN has been around for about a decade in France um and I think that it just it gets relied on even more for those three purposes and essentially just to be like we can do more with less and that like that's just something that's kind of driven into that system of like if you can you will essentially I mean there's obviously like there's like bureaucratic issues around like police unions and things like that can sometimes prevent that um and I think like the next talk we'll talk a little bit about kind of like the resistance of people to adopt those but from a top down perspective um I think all of these three goals like it's cliche it's like tautological and I think they're just like every like the people that are making decisions are like totally okay with that they're just on board unfortunately it's really interesting talk um I'm a historian of slavery and I work on information systems on slave plantations which are actually very large and you could say I work on the way soft power there is supporting hard power through watchmen and information um so that point that you make at the very end is of course extremely familiar to me and what I'm trying to get my head around I mean you mentioned a bunch of times and actually probably all of the talks will kind of comment on scale like how the scale of the data is new and I'm wondering how the scale changes that relationship between soft power and hard power because I'm working on you know kind of small data where that's happening um in really that relationship is really visible you can see the hard power really easily and if you look even a little bit you can see the soft power so I'm wondering how the scale of the data changes that connection yeah uh that's a good question um again I don't know if I can like um super confidently give you this answer but my my hunch with this is that um it reduces the amount of time that the hard power needs to be um sort of like used or like expressed uh so one of the things that I didn't quite get into in this talk was sort of the timeline about the state of emergency in France is the first like two months after the law um it was basically like no holds bar just like whatever you need whatever you want like let's do it we are in a real crisis um and about seventy percent of the house arrests were made in that in those two months um and uh one of the reasons for that is because so like now they've sort of implemented laws that have kind of been like okay you can't actually take everyone's data right away like you need to prove that they might be part of a like involved in some sort of terrorist conspiracy or criminal or like criminal conspiracy something like that in order to actually see the data like you can take it but you won't be able to access it unless you can prove to us that this is like something is right like something kind of fishy here but in those first two months there was none of that and that's when like seventy percent of all of this stuff happened so I think that um it just like allows for um the kind of like the period of um hard power like it just shows itself incredibly um powerfully and just in like a more condensed amount of time because the software is capable of just dealing with the mass stuff that you collect later and in a way that people feel feel again people feel is reliable right thanks uh for the interesting talk I have a comment and a question the comment has to do with your observation about the police seizing lots of data and they're not really doing or not appearing to do anything with it this is very reminiscent of something that um Feldman in March described in a paper from ASQ in nineteen eighty one called information as signal and symbol where they talk about organizational information gathering and a lot of times there's uh purported gathering of information to make a decision but then the information isn't actually being used so it's not entirely surprising to see that practice uh sort of reiterated or perpetuated here the question I have has to do with the description of visualizations as cognitive shortcuts which I found really interesting because it sort of has this dual implication on the one hand shortcuts suggest something that we do all the time and it makes it easier for us to do it right but in the very next breath you then say that it's a cognitive shortcut because it allows us to see information or a new and different way and fundamentally I think you're drawing out a question that a lot of us are going to be asking which again has to do with this issue of scale do these algorithmic systems allow the enactment of practices that are already occurring but more rapid and larger or is there a qualitative difference where there's actually a new kind of practice emerging yeah I mean I think that's that's it right it's like the the when does when does quantity become change into the sort of like the qualitative one does yeah amount of quantity change into a qualitative shift and I think that maybe going back to one of the first questions it has to do like it's implicated in the type of in the type of visualizations and in the type of software that's used I think a lot of the time so I I think that the the qualitative shift comes from the software from the type of visualization that you get so these type of sort of we're not on there I mean these type of Lincoln node graphs sort of like social network analysis at least to me that's kind of the qualitative shift and that the quantitative shift can kind of like build into that once it's already there because I mean like Lincoln node graphs are they haven't been around forever there were like ways of seeing and visualizing data sort of long before them but I think that at least in my example that the kind of the qualitative shift comes from this type of visualization that creates the shortcut in a way that allows that basically kind of like tells us that we're seeing things in a way that can be maybe more digestible just like because of the amount of data okay I pick up a last question while maybe Angel can start setting up our computer so we just say thank you you ask about a very huge question about algorithms and governance and you answered with a very specific case of France and I wonder whether you see you know the French context the French government is very different from the American one for instance and if you see you know comparatively interesting particularities of this use of this tool within the French context as opposed to the American for instance was that more about the if I if I see similar things happening in the United States yeah okay yeah well I mean I think in this is not only in the United States but I mean stop and frisk is essentially this on like an individual scale and I mean I think most of different kind of like policing tactics where you can create some sense of probable cause in order to then you know either shake someone down see what's on their phone or anything like that to kind of get that information I think those types of those types of policies become more important as people have more sort of like vital and crucial information about their identity on their persons so I think that like the amount of information that you had on a cell phone ten years ago probably wasn't as significant as the amount of information you can access from a cell phone today and I think that sort of those shifts in what a cell phone represents or what kind of access a computer has all of those things kind of heighten the necessity for these from the states perspective of these types of policies so I think yeah things like stop and frisk are are sort of like endemic of that same type of relationship at least in the United States but you know that also happens in France too but yeah I mean I would just I would also like look at other types of sort of states of emergency or legal exceptions that kind of focus on the extraction of data as being kind of important to this whole system thank you so much thanks everybody walk away with the mic here okay so our next speaker and the concluding speaker for this panel Angel Christine from that university down the peninsula Stanford yeah okay is that the difference between Stanford and Berkeley was so great but there is a big difference yesterday I got into a so my heated discussion about football game yeah I've heard of that yeah okay algorithm in practice comparing journalism and criminal justice okay so first thank you very much to the organizers for organizing this great conference and also for the presenters on the panel I think the different papers are very complimentary and so that's really exciting so today I'm going to talk about algorithms in practice comparing journalism and criminal justice so it's now a facial we live in an era of data, big data, small data, lots of data and there is this unprecedented amount of information being collected, analyzed, installed about what people do what they think and what they buy in particular there has been the multiplication of algorithms in highly skilled fields from finance to healthcare, public administration, policing and justice the non-profit sector, education the media etc and so in most of these fields algorithms in fact are doing two different things right on the one hand they are reconfiguring the ways in which professionals make decisions about cases they are here to complement and sometimes replaced as we've seen traditional ways of making decisions ways that were informed by some kind of technical knowledge be it low medicine or economics and second the algorithms also are used to measure the performance and productivity of the workers themselves in quantitative terms, number of patients number of cases, number of arrests etc now what I find interesting is that the discourses used to justify the adoption of these algorithms in all of these different fields are almost always the same in fact algorithms are always described as this rationalizing force in the Weberian sense and in fact if we look a bit closer we see that there are two different versions of this argument one which is about efficiency right algorithms are just better than humans at computing large amounts of information they're faster, they're cheaper all of that kind of stuff but there is also a political version that is interesting and which is really about objectivity and here it's the idea that algorithms can cure professional fields from long histories of ignorance, bias and discrimination and that they can help make experts more accountable by providing neutral recommendations that experts have to take into account both arguments in fact can be described as instances in the belief in the superior value of mechanical activity of a human judgment now in parallel to this multiplication of algorithms there has been a pushback obviously and that's why we're all here today in the large part scholars have cast doubt on the mythology of the superior intelligence associated with machine learning and algorithmic techniques big data is not only a technology but also a cultural and political form there has been the idea of black boxing right the idea that algorithms instead of making people more accountable makes them less accountable that no one understands how algorithms actually work that they're opaque for a lot of different reasons and finally there has been what's called the problem of fairness which is the idea that instead of minimizing inequalities in fact algorithms tend to reproduce and often reinforce them by creating feedback loops and that's for example Barocas and Selton O'Neill's arguments now in this debate I think that there is one question which has remained a bit unexplored or little explored and that's a question of practice we know little about how algorithms are actually used by the people who are forced to use them right or who are asked to use them to increasing extents yet there are good reasons to pay attention to practices right here we can learn from science and technology studies and the fact that technological artifacts are always open-ended and really kind of take their meaning and function from the alignment that they get into with like specific institutional context and local practices in other way we might expect a gap structured by social forces between the design of the algorithm right it's intended effect and what actually takes place on the ground so that's what my project is about and what my research program really is about so in my work I study the ways in which technologies of quantification are used in expert fields and specifically I'm interested in the following questions how do professionals use algorithms what kind of meanings do they give to algorithms and when and why do they manipulate algorithmic techniques in ways that are very locally defined to explore this question I compare fields what I call expert fields during on Bourdieu that have different characteristics in fact strikingly different characteristics web journalism on the one hand and criminal courts on the other hand so now these two fields really couldn't be more different one is private the other one is public one has a strict kind of jurisdiction of its professionalized areas the other one doesn't one is very data-rich right it's online so that would journalism or the other one has trouble with technology that would be criminal justice and so I've been doing ethnographic field work in these two sectors for now more than five years so let me start with the case of online news there are many differences between print and online journalism but what's interesting for my purpose is that most news websites now rely on advertising revenue as a unique source of income or as a major source of income and so this in turn has led to a change in the kind of you know goals of newsrooms which is that now they really want to get traffic and traffic has become the absolute priority for most newsrooms as a consequence new tools have emerged to track what readers are doing in real time so these tools are usually called web analytics software programs it's a booming market there are like more than 15 different programs that track what readers are doing and so here let me just give you an example this is chartbeat and it's one of the programs that's the most widely used in the US and elsewhere so it's used by more than 80% of newsrooms in the US and that you can see it's like it moves in real time right which is kind of addictive weirdly enough and it gives a sense of the number of visitors on the new website at any given moment how that compares with the number of visitors a week before a ranking of the most popular articles so these articles here it's not a real website I mean I think it's a real website but it's not a serious news website and then the sources of traffic direct links, social search etc. now these software programs are used on every single computer in web newsrooms journalists have to look at it basically it's installed it's there they're encouraged to look at it they receive rankings of the most popular articles etc. now what's interesting is in looking at how these companies like web analytics companies kind of sell their products is that they explain that they want to transform the relationship between journalists and their public they're not saying that oh we want you to get more clicks no no no no for a long time journalists have been making decisions about what's newsworthy without taking into account the preferences of the public now we are going to change that by providing solid, reliable data and giving them the tools to understand what the readers really want so in other words what these analytics programs are trained to do is to make journalists accountable not to their peers but to their readers right so it participates in this kind of rationalizing process that I talked about now the question is does that work so are they really changing the ways in which journalists interact with their readers so to explore that I conducted a multi-cited ethnographic studies of web newsrooms in New York and Paris between 2011 and 2015 I'm happy to talk more about it but first I'm going to talk about the criminal justice case and then I'll move to the findings ok so now let's move to a very different kind of context criminal justice so thanks to the previous presentations I actually don't have to introduce what risk assessment tools are doing but let's just say that you know it's a big deal right now they're multiplying across the US and in Canada and now in Europe as well are using these tools to a much higher extent than before and these tools usually adopt techniques that come from the insurance sector so actuarial techniques to assess the risk of recidivism of defendants based on the number of socio-economic and criminal history variables so let me just give another example it's a good thing I didn't pick LSIR or Compass because you already talked about it this is a more like grungy type of risk assessment tools it's local as you can see it has this weird like icon from the side like the eye the scale I mean it's just a bit strange and here you can see that you have you have to like you know document desirable then you can calculate risk and it spits a risk of ranging from 1 to 10 now once again so why are criminal courts adopting these risk assessment tools well in fact for many of the people who believe in risk assessment tools these algorithms can cure the criminal justice system from its bias and discrimination mostly it's left leaning advocate like reformists who are trying to put an end to mass incarceration who are advocating for the use of these risk assessment tools to identify classes of low risk offenders who can be released and not put in jail right so there is this idea that algorithms are just less discriminatory than judges and prosecutors that machines can help us be more objective right so they use a bunch of metaphors like money-balling justice using smart statistics to reform criminal justice of course it's a bit more complicated than that and we already saw like very good reasons for why risk assessment tools are in fact biased and come with a specific description of what defendants are and who are they are going to be and they raise a lot of legal social questions right as a recent article by Julia Engwin has shown and like many others now it's a heated debate about are they biased are they not biased I mean anyway there are lots of problems here that said there are very few studies of how these risk assessment tools are actually used in criminal courts and so I decided to do ethnographic fieldwork once again in several criminal courts one in the south of the country and one on the west coast and doing interviews with judges prosecutors probation officers clerks court administrators to understand how exactly they use risk assessment tools in their daily work okay so now let me turn to the findings so I did ethnographies of newsrooms and ethnographies of criminal courts and in both cases looked at how algorithms were changing the daily work practices or not of the professionals involved right so what did I find so I'm going to start with the similarities between the two cases right so criminal justice and journalism so the first similarity is what organizational sociologist called decoupling so there are many cases of decoupling in how algorithms are used in these institutions so what does that mean well it means that there is a large gap a significant gap between what these organizations say about their uses of algorithms and what they actually say so when you talk about when you talk to editors in chief or like judges or like top administrators in criminal courts are like oh yeah we are so data driven we love algorithms they are really changing we are on top of it we are shining a bright light on the whole system so they are really like yeah we are just doing it that's what's happening but in fact when you spend time in the organization what's going on on the ground is very different and that in fact many of these techniques are not used people refuse to log in when they log in they refuse to look at the scores I mean that you see basically this kind of lack of it takes a lot of effort to make people adopt new technological forms right it's just not easy at all so this leads me to my second point which is that there are many processes of resistance both in web newsrooms in criminal courts in how people on the ground so that would be kind of staff writers and you know judges and prosecutors the kind of people who are actually doing the work like everyday not people who decide oh yeah we are so data driven there are many processes of resistance taking place in fact there are three kind of main ways in which people can not resist using algorithmic techniques the first one is food dragging right so that's just like no I'm not going to use it if I can avoid using it I'm not going to use it and then like you know they use many arguments to justify why they don't use the algorithmic techniques like well I'm not good at with computers I just you know I'm just not good at it or like oh you know it's really an awkward system it doesn't work well so it's faster if I do it myself so you know there are all of these food dragging strategies there are also instrumental and gaming practices which is that both journalists and magistrates know very well how to get the numbers they want to get right so like if they want to get a low score for a defendant whom they think shouldn't get a high score because he has all of these like attenuating circumstances they're going to find ways to change some variables which they know have a lot of weight in the kind of risk assessment tool to get the kind of results they want to get and I've seen that documented many many times and it's the same in journalism and I can talk more about that if there are questions and the last kind of resistance strategy is just open critique and many magistrates many local legal professionals and journalists are just saying well these algorithms are bad, they are poorly constructed, they provide bad incentives that's not the way I see my work and they just refuse to do it for that very question okay so let me turn to the differences between moving to the conclusion so in spite of these similarities between the cases of criminal justice and web journalism there are also many important kind of differences between the two contexts in how professionals make sense and interpret algorithms and here I'm going to be a bit fast but basically for journalists web analytics are this very ambiguous tool because on the one hand they kind of signal market pressure and so as a signal of market pressure many journalists kind of resist them saying like oh we shouldn't cater to the public, we should trust our own editorial judgment but at the same time analytics also say something about their impact in the public sphere they also say something about how they shape public opinion and because of that journalists have trouble saying well web analytics don't say anything well they do say something about whether your article found its audience so it's a bit hard to resist whereas for legal professionals the resistance is much clearer and that's because there is a deep distrust of innovation in many criminal courts and this idea that really traditional practices are based on the role of the precedent and these algorithms have not been vetted by the role of the precedent, they're new, we don't trust them why should we use them when we've been acting as judges for the past 30 years so it's a much more univocal way of interpreting the algorithms so let me just conclude very briefly in this presentation I compared how algorithms are used in web journalism and criminal justice and I showed that they are both similarities like decoupling processes and resistance strategies and more generally a gap between the intended effect of algorithms and how they're actually used on the ground but also differences which is that algorithms and analytics are integrated into existing professional arrangements and routines which themselves depend on the structure and the history of the field under consideration like algorithms don't have the same effect everywhere that seems a bit simple but I think that taking different examples shows that well and so I think that adopting this kind of comparative perspective is now crucial when these algorithmic techniques are circulating across sectors at a much faster pace than before so we need to understand better how people resist these techniques in different ways depending on the local context and I'm going to stop here thank you for the fantastic talk I'm wondering if you could talk a little bit about how previous instances of resistance to new technologies in different sectors might not predict but condition what we think is going to happen with some of these algorithmic resistance strategies because so Karen Levy's work on truckers and electric monitoring also is kind of a good case of this I mean the resistance drags out the process but there's can be a kind of inevitability that comes from a kind of top down regulatory or technological fiat so I'm wondering where you see these two cases going yeah so listen yeah that's a good question and that's where I think long term ethnographies also matter to see how these situations change over time because I really think they do change for example journalists used to be super resistant and they really used to hate analytics and now analytics are just part of the job and they're also part of the career incentives so like now for example when you're applying for a new job as a journalist you have to give your number of followers on Twitter and so you know it's a kind of small things but like basically when it becomes really enmeshed in the careers themselves it's because much harder to resist I mean academics are also a good case here and for criminal courts my impression is that there is a lot of backlash against risk assessment tools right now and courts are very conscious of that for example all the people I've talked to had read the ProPublica article so I don't know I wonder if quantification in the case of criminal court is not going to take another road which is the adoption of digital case management systems which is actually a much bigger deal for much courts so anyway it's a longer question but yeah I have a question about the journalism part of your research and one of the things you mentioned is that the way these tools are marketed to journalists is that they try to change their field so from being responsible to their professionals they are now responsible to the public and I was wondering if the adoption of these tools depends also on sort of the generation of journalists so for instance I've read that the New York Times editorial board thinks of themselves as setting an agenda whereas the breed of journalists for instance like Vox.com their founder believes that they are going to explain the issues to people so given that they have these widely divergent aims do they use these tools differently so I was curious if your field work is more than anything absolutely and so actually like so the journalism part was my dissertation research and I studied different types of newsrooms from kind of legacy newspapers to like more like news aggregators type of things and basically the way in which the metrics is very different I didn't talk about that but basically legacy newspapers have a containment strategy where they don't give the metrics to everybody because they feel that that's not relevant to how journalists should do their work whereas like you know aggregation type websites like the Huffington Post or Buzzfeed or you know Gokker used to be are much more metrics oriented so there is a lot of variation for exactly the reasons you describe and also for a reason that's why she's size of the organization Hi, I have a related question that's methodological I was curious about the logic behind the comparison here because it would also be interesting in order to control as many of these factors as possible if you compared for example one journalistic field where algorithms were not really relied upon in practice they were traditional whatever that might mean versus some other field in which they were like heavily dependent on algorithms for business model or they were like whatever right and so I just like to hear you respond to that and ask like yeah how do you what do you think you would tease out specifically about what algorithms are adding to or subtracting from the practices of a specific social space yeah so so that's a really good question I think actually I'm thinking of adding a third case so you know but yeah basically I mean I think that here's a kind of comparative approach also there is a question the comparative approach in this case is to take fields that are as different as possible but are both becoming data driven right so like I'm interested in the transformation and basically how this transformation takes place when all the kind of structures of the fields are just so different that like we could expect like that you know they would turn to data driven techniques in different ways that said like and that's actually a really good point it's like as a third case would I want to take say like I don't know marketing like departments which have always been data driven like things from you know distribution commercial spaces like you know that's like yeah anyway I agree that the comparative question is a good one thank you very much for your talk I was wondering these two very performance driven practices the one of journalism and the one of justice in this setting algorithms affect two things that appears to me quite relevant the one is how something becomes a fact in the case of journalism and in the case of justice how something is proven so the proof and effect on the one hand I see it very connected in your ethnographic account and I was wondering if you reflected on this connection especially in an STS perspective or in other perspective thanks so could you just expand the tiny bit on what you mean by becoming a fact yeah I think the news are accounting for something that happened so in this case there is a fact but this fact is built through algorithms and the performance of the journalist practices is related to what becomes a fact and the performance of justice is related on how something is proven and I was wondering if you reflected a little bit on this connection and how algorithms are affecting in this especially organizational culture based on performance what happens to facts and to proofs yeah that's a very interesting question so let me just like answer in a bit of a kind of diverted way but I think it will address your question I think that one of the big differences between the two types of software programs is that in the case of journalism the major articles that have already been written right so the product is already there so question then becomes how to promote it how to retitle it what kind of headlines should be there how to promote it on social media platforms etc but the article is already there and they're not using actually there is a whole branch which is called computational journalism but like right now that's not what I'm talking about these software programs are not participating in the elaboration of the news just in how they're managed once they're produced whereas in the case of criminal justice it participates in the actual decision making process about very specific cases so I think that the relationship to facts is a bit different just because of the timing of when the algorithm comes in in the kind of production process to use a kind of you know industry type of term that's great work and I've wondered about I was captivated by the foot dragging point that you were making and wondering to what extent this your findings in this case link to a more general issue about the introduction of new technologies of efficiency in kind of labor situations and whether one of the differences another one of the differences between the journalism case and the criminal justice is that I mean I'm curious with criminal justice because with journalism you could say the management is introducing this as a means of improving the economic performance whereas in criminal justice like who is driving it and then if you were to take another case the question is whether the introduction of the algorithm is a way for management and maybe the people who are on the ground people have purposes and the algorithmic intervention could be a way for management to calibrate or recalibrate in a way that will push the people underneath them to accept and go with the goals that the management has and then what's going on in the criminal justice system so this is thank you Helen for this great question because I think it's actually one of the fundamental differences between the two cases which is that in the case of web journalism basically everybody is on board with the goals that the publications should not die because all the journalists are going to be out of a job if it happens right so there is this kind of like common incentive which is just like yeah like we understand that it's a competitive market and journalists are very conscious of that I mean a bit less at like big organizations because they feel more protected right but like as soon as they're in the kind of start up new media world they're just like yeah like we understand that we need to get clicks like that we understand that that's a goal so they are on board with the ultimate goal now the question is like should they be the one doing that work or should it be the role of the editors and that's usually how journalists defend themselves they're like listen I'm a writer why should I be taking care of traffic it's not my work it's the work of the editor after all they're paying more for exactly that reason right so like but I agree well as in the case of criminal courts obviously that's not what's going on and I think that it also explains that the resistance is much higher it's because usually so these algorithmic tools are introduced so depending on the case they come from different places right they either come from a small department so from the probation department the kind of parole board of parole etc so they can be kind of local initiative so I think oh yeah we have to do that but they also come often from the kind of central administration of the court and usually from the technology office which are playing more and more important roles now in criminal courts because there is this big challenge of like how are we going to manage our cases and how are we switching to paperless systems which like is a big deal and so usually it comes from this kind of administrative department in criminal court knowing that judges, lawyers and prosecutors have very little respect for the administrative functions because they're like no no we're real thing they are just leeches on the system you know what I mean so so there is not at all the same kind of incentives and there is not this idea of like that we're all in the same boat and in fact judges and prosecutors feel often not always but that the algorithms are trying to replace their own kind of cognitive models for what risk is and they feel like really attacked and under threat for that reason whereas journalists are like well it's true I don't know what the readers want I wouldn't have that information otherwise so yeah thank you so much so we can continue the conversation of our break I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I