 So I'm already a bit older than most of you so that means I grew up in A world where I was the only kid in my town who liked computers and Star Trek and like Hitchhiker's Guide to the Galaxy and weird shit and being here with all of you is just like having friends So I really appreciate All of you for being here So my name is Joss. I got I'm a software engineer at some scale up in Silicon Valley and I have a law degree And so this is me and my lawyer's robes and I'm gonna talk about predictive policing Specifically predictive policing in the Netherlands and so we're gonna talk about What is predictive policing? How do these systems work? I have some examples of predictive policing systems that are deployed in the Netherlands or were deployed in the Netherlands Specifically, I'm gonna talk about the legal status of suspicions generated by a computer and the question there is If a police officer walks up to you and goes like I have this print out here And it says that you probably up to something no good Can they arrest you for that? And so this has never been tested in court But I'm gonna predict this for you. And so in a few years you will see that I'm right That bit includes audience participation now in order to participate in the audience participation You need to be able to raise your right hand. So all of you please raise your right hand now Okay, most of you are able to do the audience participation bit. Thank you very much So introducing predictive policing. What is predictive policing predictive policing is? Generating predictions about crime with a computer, but that's it basically this can be crime Committed by persons or crime that will victimize particular persons or it can be places where particular crimes are gonna happen This is a The term predictive policing was coined by Bill Bratton who is an outstanding police officer Who was the chief of the Boston police force? The LA police force and the New York police force. So this is a guy who knows something about policing the term was Use it doesn't really apply to a particular algorithm So I am not gonna bore you with the question is this artificial intelligence is this big data is this machine learning Is this risk profiling? I don't particularly care if it's a system that uses a data and a computer To generate a prediction. I call it predictive policing Because most of these terms mean different things to different people. They're not very interesting What is big data for a police officer might fit on like? Sort of the spares basis between the memory buffers of my screen somewhere Artificial intelligence seems to be a term that is only used for things that only humans could do until recently But now a computer can do it as a time when playing a game of chess was artificial intelligence And now this is no longer true. So this this is not particularly Interesting. I'm going really fast. Are you still with me? Awesome, thank you So We have been predicting crime since the dawn of time Because you know see the dawn of time that was crime and so people have been trying to predict that and even before we had computers We had people that were trying to predict crime and they're called crime analysts and every police force has Crime analysts and these crime analysts they use crystal balls Calculators An abacus. This is this room is called abacus. That's why I had to say abacus To generate predictions about crime. So if we're doing this like since the cows came home but then happened then computers came and So computers came and people thought what are going to do with that the answer is play pong But then Bill Breton Ran a program called comp stat who here has heard about comp stat a few people anyone here. Did you see the wire? Yeah, okay the wire comps okay comp stat was a program in the US police force Where they looked at data and tried to steer the use of police resources people cars horses dogs pepper spray to optimize clearance rates and Clearance rates is like just crimes being solved and so these people had like weekly meetings And they were like hit over the head with a police baton if the clearance weights went down I said, oh my fucking god the clearance rates need to go up. How do I make the clearance rates go up? Answer is easy stop recording crimes Because if you stop recording crimes, you have fewer crimes and then or you only record the crimes when you have solved them Because then the clearings rate clearance rate and the crime rate go hand in hand I can really recommend that practice if clearance rates is the only thing that you are interested in Bill Breton thought like we have computers. We can use these same stats and Excel and Generate predictions about crime and this is what happens basically I'll get to that later how that worked But modern technology added and I'm not going to talk about machine learning a lot because all of you know how machine learning works And all of you know how tensor flow works and all of you can at least do sort of logistic regression on your HP calculators Computers added the fact that quantity has its own quality so if you have lots of data and you have lots of processing powers you can go to a level of Finding patterns in that data that are impossible to do if you're just a human humans are not Very good at finding accurate patterns in large unstructured sets of data And so I got this quote here that says humans can sometimes find a needle in a haystack Modern technology can search all the haystacks Figure out what's normal in the haystack find all the things that deviate from that norm which might be needles And he gets who said that Yeah, I said that so thank you very much It was in my master thesis on this very topic and because my master thesis was evaluated by law professors and Law professors and computers Don't mix very well This has become so standard that you can buy this book Which is called data mining and predictive analysis and if you want to have like a little shrill moment This is scary You need to go to Amazon then you have that shrill moment But then you need to continue and read the description of this book and then you get your second shrill moment because This book describes how you can use off-the-shelf tools like Excel To find to predict crime So the author basically says here's a shit ton of math. Here's a bunch of Excel macros Export the CSV file from your local crime reporting system if you're down in Hackensack Police department and you can use this and now you can predict some crime and police officers do this This is extremely scary because these people don't necessarily know what they're doing when using computers or not using computers Okay, are you still with me? Ten minutes in this is going well. So How do these predictive police systems work? They work like this is a diagram that I copied from my Research report on the topic and it basically says you have some data You run some math Now you have a prediction and you give that prediction to a police officer and you go like here's a prediction about some crime Go do something with that and then they find crime And they have new data and they run the next generation of the model and then you have some new report And this is literally how this goes so that you can see the feedback loop here where actual data turns into historical data and turns into the next generation of the model and this is one of the problems with Predictive policing and I'll get to that later, but I'll announce this now quickly all of us as we are sitting here are Committing some crimes And the reason for that is that the stack of law books is about this high and There is no way that any of us are in complete compliance with everything that's in there The only reason that you're not arrested right now is that you have not annoyed enough of the right people To be arrested and there's just not enough police around here in uniform This is a tip for people if you want to discover undercover police officers in the Netherlands Look at their shoes because they're all wearing black Mephisto's with shoelaces So if you see any of these people, they're undercover police officers, so you haven't Annoyed enough of the right people yet. That is why you're not getting arrested because you're all doing something wrong now This means that if you use predictive policing to send a bunch of police officers to a place They will find crime there Because there's crime everywhere and then they find that crime that goes back into the model and go like oh my god This is a place where there's a lot of crime that sends some more police officers to there So they go back to that same place and find more crime And this is the feedback loop that a lot of predictive policing Systems are vulnerable to I'm not saying that this is happening all the time, but they're vulnerable to it Let me see that about my speakers now say No, I got it all So let's talk about some predictive policing systems. I I named for I'll cover three Very briefly. There is information on the internet about what all of these Systems do in some more detail So here are some predictive police Systems first of all Pratt Paul From the US also applied in the United Kingdom The strategic subjects list, which is one of my favorites, but I'm not going to spend a lot of time on it But like if you want to know more about it talk to me today or tomorrow because it's fascinating ship The crime anticipation system or as I say in Holland had criminalities anticipation system Leukwoord for Galchie or for scrabble 48 punten And sensing which is also in the Netherlands Any of you have a favorite system that's not on the list No, okay, sorry Free crime. Oh pre-crime. Oh, yeah. Yeah minority report. Yeah, I was I was like this close to call my Massethesis minority report on the a because the a is the river that goes through Amsterdam But I couldn't get the introduction running It's kind of weird and so if you you have seen the movie minority report Some who has read the short story Okay, it's fewer people. Okay, all of you who have seen the movie go read the short story like any of standing of Like any of his work. What's the author again? Sorry, oh, yeah, Philip K. Dick like any of thank you very much any of Philip K. Dick's work the stories are infinitely better than the movie and all of Philip K. Dick's work You have to answer the following three questions or now actually now you have to answer one question The question is who here is on LSD? Is it the reader? The protagonist of the story or the author Potentially more than one Like that it's and so minority reports stunning. It's a decent movie if you like Tom Cruise It's a stunning short story and it's about predictive pleasing using mutants the police in Holland does not have telepathic for Mutants with foresight, but they have computers However before we get there. Are you still with me? Yes. Okay. Are we enjoying this? Thank you First first after Pratt Paul or as they name it now Geo litica So this was based on the insight that I mentioned before which is that the data that you get for comp stat Can be used to make crime predictions and so they put some scientists on it It did live wet white lab coats from UCLA and Santa Clara And they developed an algorithm and that algorithm is called it as and that is patented in Patent warp you can read it. It's like any patent. It's a completely useless It doesn't say anything about anything and there's a Commercial implementation by Pratt Paul Inc. Which is now renamed and they sell this through the United States And because it is a company it is secret how it works We don't really personally. I think that's a problem But I'm sorry. I work and live in the United States And I'm the I'm the odd one out there So we're in Holland for those of you who follow Dutch politics I'm like on the right side of T66 Which makes me sort of like a suspicious right-wing character here in Holland in America. I'm like a fucking communist People come to me go like yes, you don't understand Bernie is a socialist and like no no no Bernie's a millionaire Socialism is the common ownership of the means of production And then I've lost them I've said he Anyway, so I think that is problematic, but as we will see other Systems aren't much better So what are the foundations of the ETA's algorithm the foundation of the ETA's algorithms are three? One of them is called repeat victimization It basically means if you are a victim of a certain type of crime You are more likely than most to become a victim of exactly that same type of crime again Secondly near repeat victimization People that are near you if your house gets burgled the houses around you have a higher probability of being burgled within a short amount of time Because burglars work street by street Let's work street by street or they take the same bus into the suburbs Rob a bunch of houses the next day they take the same bus into the suburbs that break into a bunch of houses This is one of the reasons why we don't have public transport in the United States because I am not fucking kidding you People do not want subway lines or buses into their neighborhoods because that will bring criminals From it were in case of Boston downtown crossing. We just like right downtown. We're like three major subway lines meet and local search and local search is the most Amazing inside of this is that criminals are very likely or not are more likely than average to commit crimes Near what we call the anchors of their existence. So their home their work and on the route between them Personally, I think if I would be a burglar I would not burgled the homes where I live because that makes it much more likely that I get recognized but Apparently burglars do this now This is not crazy shit. There is actual science that sort of bears these statements out So that's criminologists. Are there any criminologists in the audience? No, no criminal so criminologists they studies a crime as a social science They've been writing papers about this for a long time. And so this basically this bears out This is this is true for the to the extent that you can say that things in are actually true So they developed the ETAs algorithm epidemic type aftershock sequence and the principle is okay There's like a chance of crime and because of these three patterns They're sort of an aftershock like an earthquake and that aftershock dampens over time And then they have this formula that comes from the pret poll website. They don't say what any of these variable means So it's called it's called proof my intimidation Intimidation matter. Oh my god. I've got math. This shit must be good like so And this is what they do So what I would love to discuss but I have no time for the field experiments They did that showed that has all sorts of like problematic reasons and pret polls amazing statement that their system is free of prejudice Okay, that was pret poll system number two sensing Sensing was a project by the Dutch police of Roermond. He just said that in the south of the Netherlands Roermond has a big shopping mall and they had figured out that Criminals come there to pickpocket and rob and break into cars. It's called Mobile Banditism Mobile Banditism And again like you we're gonna do We're gonna develop a predictive policing system that will just tell us Who these people are and then we're gonna deal with them before they get to the mall This system which has many problematic aspects was paused in 2020 because what happened in 2020? There was a sharp decline of people coming to the mall one of the great Advantages of the pandemic that this system was killed. So how did sensors work? We don't know it is secret There was a freedom of information act Request for Dutch the way obey or as you call it now the way. Oh, oh the vet open overhead And they figured out that it's probably related to the following things There's a group of men in a car with a German or Romanian license plates rental car three to five years old or a car that stolen That's probably hugely important and then it goes into the secret these and other factors go into the secret algorithm Projects a risk score if that score exceeds a threshold it goes to some police officers on the street or like on the freeway in this case and they go and stop that car now the question is based on the what? legal Theory and I'm gonna explain to that. So let's stay tuned. But that's what they do The freedom of information act gave us the following information. You see you see there's a little n here That was not secret So the letter n Has got something to do with this I mean some of the shit that they blocked out is amazing. This says the administration of the reference lists is taking place at Locked out some office secret office or The responsible department for the downloading and updating of the reference file We can't notice Fabric of society would unravel when we know this shit Third one up the crime anticipation system that criminalites anticipati system developed by the police unit of Amsterdam Currently being rolled out and implemented in the entire country The goal was to create enriched forecasts of crime and turning these into tangible instructions and then ensure that beat Plot beat police officers act according to these instruction. This came right out of the Dilbert missing statement Generator, there's amazing video on YouTube crime costs for kids and explain sort of like in to kids like you know This is your computer and amazing video How does it work pretty much the same as it has so it has focus on burglary and street robberies it Takes data from all sorts of sources. It generates heat maps on where the crime is going to happen in squares of 125 125 meters and then the 3% of squares with the highest risk scores are called yellow orange and red and Then you get something like this This is a Amsterdam Apparently a lot of crime in the east of Amsterdam without the computer. We would have never guessed this Completely novel insight strangely enough Not on the map, but there's something here in the south of Amsterdam called the southern axis where all the big law firms are Not colored red, but I'm pretty certain that there's a shit ton of crime Happening right there But that is not a crime that they're interested in these systems exist I will Keep my social commentary to myself Some issues around predictive policing now This is a very suave audience that knows about computers. I'm gonna like run through them because it's completely obvious First of all input data quality crime reporting is Incredibly hard and criminal data about which crimes happened and where they happened is completely unevenly distributed based on Social economic factors type of crime Who is liable to report these crime like there's a hope it's it's a morass. So a big thing in Criminology, which is the study of crime as a social science is guessing the dark number the unreported crime I've never heard any discussion about this in any of my Predictive policing research that I've done please seems to be unaware of this Secondly This is the only thing I'm gonna say about that But apparently there is some racial prejudice in society and apparently computers leverage and amplify that racial prejudice The police seems to be completely unaware of this never read anything about any of this in any of the research That I've done including reports by the police itself Personally, I think this is a problem But again, what am I am white in America? And so who am I gonna complain to? So the social justice impact is amazing these systems get used to predict low-level street crime Bernie Madoff was not caught by a predictive policing system I know for it's always big pocketing Street like burglaries Robberies, etc. And I already mentioned the feedback loop Most importantly does it work and the answer is we don't know Whether it works. What I do know is that a few teams in America Discontinued to use of pretpol because either it was too expensive or They just couldn't figure out what to do with the predictions. So he said does it work? Yes, the predictive aspects some of somewhat work, but I think that they're also obvious like Does it lead to more solved crimes? I have not seen any definitive data about that. There are some Reports that say that it works, but they're mostly done by people that are sort of not unbiased with respect to the police predictive policing system that they're using Does it then at least lead to the same clearance crimes with fewer cops? Even that I don't know personally. I don't think it works Right, but okay So that's that now we get to the legal aspects of predictive policing Let me take a short break here to ask you are you still with me? Okay, cool. Hang on. I love giving talks to a hyper intelligent audience The hardest course I have ever taught was Java for non-programmers It's a bit of a hard course because someone asked me When I had to explain the if statement, but why do you need an if statement? So I thought about it for a while and I told her like well Imagine a world with an if statement and imagine a world without an if statement That in itself is an if statement. I had a lot of fun there Now I was people were super nice Okay, legal aspects now. This is a master. You're fish are talking your friendly neighborhood lawyer In the Netherlands You can only the police can only use certain invasive investigative measures When there's a reasonable reasonable suspicion of guilt based on facts and circumstances That a crime was committed and that the target of the suspect is guilty of it For the Dutch people among us article 7 in 20 straff ordering As we thought the word before that the vervolging is angefangen A gemerkt degenen ten weens aansien uit feiten of omstandigheden een redelijk vermoeden van schuld aan een straf maar feit voortgoed Now in America we have the fourth amendment, which is a reasonable Stop and search is a search and seizure Volumes bookshelves libraries full of legal analysis about the fourth amendment in Holland the entire Sort of like jurisprudence and legal analysis on article 27 is about This thick 12-pitch single-sided US letter. I know because I printed it out And I'll explain to you why so you have to have reasonable guilt of Suspicion crime, etc Now we have to so all of you everything you've ever seen on television about this is false Even if you've watched the US television show and you think you know something about the fourth amendment. It is also false TV my friends is not a documentary on the US legal system With the possible exception of we own this city by on HBO Max, which is totally amazing So even in the US the fourth amendment doesn't work the way you think of my work read these two statements Or to read these these two court cases because the US Supreme Court has been executing a low-level war on the fourth amendment since 1984 Ask me about this after the fact I got amazing case law about this So forget everything you've learned on TV. So now it is time for audience participation by a raise of hands So this is the case the police officer gets a Call from the office that has a burglary in a school and then he walks around and he sees someone he knows professionally Walking from the direction of the school wearing black gloves. It goes like that. That's not kosher Subsist person and arrest them On suspicion of having committed that burglary was this reasonable if you think this is reasonable suspicion of guilt raise your hand Okay, none of you should ever talk to please officer ever. This is reasonable suspicion of guilt In the Netherlands famous case exactly this case look for zwarte handschoenen Inbrake and you'll get the case. This is reasonable suspicion of guilt next one Sorry Now if you just look for zwarte handschoenen In brak school Then then you can find it, but it's it's exact. It's exactly what happened if you read this you got the summary of the case Apart from the fact where the judge says like no, this is totally reasonable I I'm not I'm not sure what I think of this. I must honestly say We don't know who knows if we don't care. Oh Oh, sorry, thank you for pointing that out you're absolutely right the question from the audience was was this person convicted I don't know and I don't care because Come again No, I'll stay at it The question is is it stand the holder or are still I mean Dutch man. It's not a language It's more like a heap of words that starts with Yeah, well My wife is American after like many years of marriage I got her to save on heart a gefeliciteer to my birthday I think this is a Lisa honey if you watch this on the stream or on video. I love you. I Do she's she's a remarkable piece of work and People who know me better go like dude. That's what you need really so the question was was it legal to just stop them or arrest them and I can't quite remember and it doesn't matter I need to go on hope second case To police officers see a black man running away from a cafe that they know is a place where drugs are being sold They think this is suspicious black man running away from a known drugs then They stop him. They find some heroin Even in the Netherlands when this was case happened in the 70s. They thought this was not cool So they arrested him Regarding the stop of search is his reasonable suspicion of guilt or not who thinks this is reasonable suspicion of guilt You're wrong. The court said it was not reasonable suspicion of guilt Black men running in the Netherlands completely legal. I mean We are worldwide known for our tolerance and this shows it Actually people think the Holland is tolerant. We're not we don't generally give a fuck It's not tolerance. It's just like I don't care. Okay Audience participation number three police officers again. See a white man Talking to a few black men Sometime later, they see the group driving around in a car with a German license plates This is why we're in Deutsche angefangen habit is the dutch is in the air for the hollandish jurisprudence The next thing that happens is they see that car parked in the street They go like this is bad like white black men in the same car German license plates This is crazy shit So they they cross the road to go to that car and the black men run away and we know that that's legal black Black men can run away But it got like that is crazy. They search the white men find drugs Recording the stop and search of the white man. Is it reasonable suspicion of guilt or not? Who thinks it's reasonable suspicion of guilt? Yeah, okay, so the people raised their hand are right. This is reasonable suspicion of guilt and now so this case is called Damrak, which is a street in Amsterdam. It's where it happened and the last case is called Used to be called differently, but I now like to refer it as to the running colored gentlemen Um, but that is not the the traditional name of the case don't ask me what it was And that guy can't survive was at the Sorry Your words not mine. So it's a go talk to this about your name lady What's your name? Okay, go talk to Inge and she has She can say whatever she wants because Free country man. Yeah, so you can ask her what is the difference between this case and the last case? We don't quite know what but what happened in this particular case And it's very important is that the police officer who did the stop and search said like well in my vast experience as a police officer in Amsterdam if I see foreign white men talking to black men That's a drugs deal and The judge goes like that's completely reasonable So this is reasonable suspicion of guilt Audience participation number four and the last one I promise two police officers see a person. They know professionally walk around with a plastic bag Hmm, so they talk to the person go like hey, what's in the bag and the guy says like oh some books. I've just stolen Criminals are not necessarily smart And so they arrest this person on suspicion of theft of books Regarding the stop and the ensuing conversation. Is that a reasonable suspicion of guilt or not? Just yes. No, who says yes Okay, okay, who says no Okay, you're all wrong And I will explain to you why you're all wrong the judge didn't get to the question Whether it was and I this requires some fine legal training Didn't get to the question whether there's a reasonable suspicion of guilt or not Because the police just like any other citizen can have a conversation with someone and just ask Hey, how are you doing? What's in the bag? That is not an execution of Investigative powers by the police It doesn't require special permission to do this anyone can do this if you now Confess to curving committed a crime now they have reasonable suspicion And they they arrest you and so There was they didn't need reasonable suspicion of guilt to talk to the Person because anyone can do that and after they are the guy had basically confessed that he committed a crime Now they could arrest him and it was completely reasonable Everyone still with me Okay, so this goes to something called the continued application of powers and this is important Remember this phrase forward to set the two passing from the voegdhede I Cut this piece from the actual case put it in Google translate and this is what came out of it Which is made I've worked for Google for 12 years. I think an amazing company If for no other reason that I can do this What happens so the police has powers under different laws Some of these are criminal laws. Some of them are administrative laws. So the police As the administrative executors of the liquor act Can come into your cafe and check your permit While they are there Legally as the administrative officers of the liquor act. They see a gun And they go hey, it's a gun. It's illegal Now they can continue to use powers that they get from another law Criminal law Because now they have reasonable suspicion This is super important. This is what happened in the last case about the books in the plastic bag this has happened in what in the the the Gewehr arrest the rifle case and This is what happens in sensing Because what happened is remember sensing remont mobile banditism cars What police would do is they would get the risk score and Then they would go and stop the car under their powers in the road traffic safety act and According to the road traffic safety act the police can stop any car Inspector driver's license and registration Like and that's come and they don't need any reasonable suspicion can just do that Like whatever and then while they do that they're gonna come this kind of suspicious These people are smoking dope. I don't like this car. It's a bit of a mess They don't have a real story about where they are where they're going I'm gonna Frisk arrest search like whatever the investigative powers are and This is continued application of powers. They stopped the car under their powers in the road traffic and safety act We have kids that and then they can once they then get like a reasonable suspicion they Continue to use the powers from the criminal law and that is how they sort of get around that now You would ask yourself. I would ask myself is this legal and the answer is no It's called the Tournament de pouvoir is like a rerouting of powers You can only use an administrative power for the explicit purpose that it is given for If you get powers out of the road and safety act You can only use this to improve road safety. You cannot use this as a cover For something else altogether, which is like mobile banditism. So it's not allowed. Does it happen? Yes all the time Like why is this not stopped? It doesn't get to court These cases the courts it's untested Like and also when in court the police lies Okay, I don't hope to please one of my nieces works for the police But please lie they go to go to go like no, no, we we just we really just you know Carl looked a bit flimsy wanted to check the insurance. You never know and then completely unexpectedly Ten minutes we found this So they do this I Think this is problematic So why does this not get tested in court two reasons reason number one These cases only come to court for guilty people Because innocent people They get like because they're materially innocent they get sent packing and go like fucking happy I'm done with this shit and they're not gonna go and press the legal matter Guilty people go like I'm guilty as fuck But maybe I can get a way out and then the court guys like yeah This is one of these things that guilty people do to escape punishment. That's really not right And so the the Dutch version of the district attorney the prosecuting officer and the judges are just not often willing to cut people a break And this is also related to a fundamental difference in the Dutch and American system of criminal prudence and the jurisprudence It's like the inquisitorial versus the accusatory or system court the goal of the Dutch system is to find the truth and Formalism shouldn't get in the way of that too much even when the police and the district attorney it opened by ministerie are like Way out of line. The judge is very very restricted in like putting a stop to this so That's part one part number two Your reasonable suspicion of guilt needs to be based on facts and circumstances Just a feeling is not enough However, the list of facts and circumstances that can lead to a reasonable suspicion is very long and very inclusive one of the few times When the Dutch police is very inclusive when it comes to Fact and circumstances This stuff is only tested marginally in court and by marginal testing. I mean that the judge asked himself Is it completely unreasonable that the police officer got to this conclusion? And if the answer is no, it's not completely unreasonable is reasonable and it's allowed so Grounds for reasonable suspicion caught in the act the normal situation. I is police officer season. It's not normal here Knowledge and experience. I know that if white men and black men talk it's about drugs official notices Tips from the team criminal intelligence anonymous tips are allowed as grounds for reasonable suspicion sometimes the court put some fences around that but the supreme court said The general thought that an anonymous tip cannot lead to a reasonable suspicion of guilt is not true So it's actually easier to say What is unreasonable suspicion of guilt? It's if it's completely based on the intuition if it's entirely based on prejudice If if it's clearly unreliable or the argumentation is just completely flimsy then it's Unreasonable if it doesn't meet any of these tests reasonable application So reasonable Suspicion of guilt So what does this mean for predictive policing said it's untested in court, but I am totally pessimistic And why am I the totally pessimistic? I'm totally pessimistic because information from the predictive police system Is a source that is based on data About crimes in the past is not that dissimilar From the knowledge and experience of police officers. It is not that dissimilar from Things about the normal situation. It is not that dissimilar to anonymous tips It is not that dissimilar from the work of crime analysts that put down their work in Official notices and send it to the police officers and we know from existing jurisprudence that all these things are An important factor in determining reasonable suspicion of guilt So I am very pessimistic that the court will say like well, this is completely that some some computer I understand logistic regression. I understand bias amplification. I understand dark numbers I understand I think this is unreasonable. I don't think the court's going to say that I don't know But I think the court's not going to say that I have some hope Five minutes. All right. I have some hope That the court will some court someday in holland Will go and go like well, I need to understand how this data gets put together And so that goes to explainability of machine learning models Now this is not a talk about machine learning But like there is there is some theoretical research in the area of explainability And that goes to questions like how did this machine come to the outcome? Now certain types of algorithms and machine learning models are explainable You can sort of figure out how the input data was constructed and the output data arrived like decision trees and most of the regression models, etc A whole lot of models these days the deep learning models are not explainable You put a ton of features in it. You have no fucking clue how it got to the outcome. No one knows I have some hope that non-explainable models will at least be Put to kibash on but I I hope hope is not a strategy The concept of marginal testing doesn't help Courts are very convinced by experts if I show up in my other capacity a senior principal engineer And I go well, uh, judge this this this model is like High quality. It's like established patterns in the space It is made according to the best practices that we know the court goes like well, you know like yaw says so This must be true I personally am worried about that that people would believe me In general But then people go like, but what about siri system risk Indification, uh, who was it? Maxim february soon-to-dutch State about like a system like this People who come to me say about siri. You're clearly not a lawyer Siri is administrative law This is criminal law two completely separate domains Have nothing to do with each other. So We hope that some of this wisdom influences the criminal courts, but i'm pessimistic One of the reasons why i'm pessimistic is that Criminal law has a lot of carve outs article two of the gdpr says this regulation does not apply to the processing of data By competent authorities for the purposes of prevention investigation detection or prosecution of criminal offenses Or the execution of criminal penalties including the safeguarding against and the prevention of threats to public security GDPR does not apply to predictive policing systems GDPR civil law completely different domain administrative law completely different domain criminal law to the point that Certain legal terms. So for the dutch amounts that word good Has a different meaning in criminal law and in civil law Just just kind of decided so i am completely not optimistic about any of this um What i am pessimistic about is the police is all in on this technology They're all in on predictive policing i see and i have seen no evidence in law enforcement circles That anyone's even thinking about this especially not in the landlands in the us I have found some papers in legal journals where there is some thinking about this in holand I haven't seen any of that and like uh, there was a there was a pilot with the criminal anticipation system 16 recommendation at the end of the pilot 15 of those were process and organizational measures how to use cost better within the police organization Who should print the form who they should give it to One recommendation about class effectiveness zero thoughts about bias prejudice Amplification legal status or the desirability of predictive policing police is all in on this shit Like so i personally am worried about this And that's all i got to say thank you very much