 All right, ladies and gentlemen, I think we still have a couple of open seats at the bottom there for those Looking for seats But without further ado get started. Good morning. Well rather good afternoon. I'm Daniel Iola program coordinator at Berkman client center Just wanted to relay a couple of announcements to you all about art some of our upcoming events on Thursday we have litigating free speech cases in the African regional courts featuring Berkman client fellow nanny Jensen Revent law so that will be in Houser and There are next lunch and talk will not be next Tuesday, but the following Tuesday November 15th And that's the end of ownership with professor Aaron presanoski of Case Western Reserve Law School so Without further ado I before I introduce our Introducer of our speaker. I just want to remind you all that this production is live webcasts for possibility It'll be available on YouTube within the next couple of days. So be mindful of what you say and Yeah, that's or not that that's an option as well. So now Thanks everybody, thanks so much for being here I'm gonna be really brief and then hand it off to Pala just by saying that I run the clinical program here at the Berkman client center We work a lot with the ACLU of Massachusetts and Pala first came her work first came to my attention through that connection to ACLU The data for justice project that she's going to be talking about today is an outgrowth of the work that she's been doing at the ACLU I am the ACLU of Massachusetts over the last year as a Ford Mozilla open web fellow And I've seen her right about her development of a data visualization framework I think we're going to hear a lot about that and about more broadly about the themes of data science in the public interest and Without further ado, I'll just hand it off to Pala. Thanks And to be clear, we'll save plenty of time for questions at the end We have a couple microphones so we can make sure everything is captured So just at the end we'll we'll hand those around and do some Q&A Thank you Can you hear me? Okay? Yeah, cool. So thanks all for coming. I'm honored to be here talking to you Especially in this, in this topic that is very interesting in my opinion, let me just put my password So the topic today is public interest in the data science and I want to start by saying that Well with this quote, which is able lawyers have to to a large extent allow themselves to become Agents of great corporations and have neglected their obligation to use their powers for the protection of the people This quote started the public interest law And I think if we change lawyers and we use technologies the same applies So this is what I'm going to talk about today And but first I want to define what is public interest, and I think it's very simple It's to affect change in public policies in the interest of the Commonwealth and to advocate for the underrepresented and Build this or cheer for the underdog so to speak Then what is data science? Data science is to track knowledge and insight from information and data and then it's the intersection of math statistics research information science and computer science and If we take that into consideration, it might be very simple to explain what are what what the public interest data science is so What happens like the motivation of why I applied for these fellowships and why I'm working on what I'm working on is because I was wondering what will happen if you put data and technology in the hands of civil liberties organizations human rights activists media outlets Governments so my background is a little bit in the government in the open data movement and The open source software movement So I I have lived like my experience is Just in a few words is that I will be here if it were not for open source and the internet and the open web so I think that Data and technology see change my life. It can also change society and I think we are going to See that more in the future So now that we recognize that there are civil liberties organizations of human rights activists that need data and technology But why what what can you can we change? Oh? Sorry, let they want me to move here. Yeah Okay, let's they need to control this Okay Yeah, this is better. Yeah, I'm fine But I have a mic here Hey, hello. Hello. Hello. Hello. Okay So What do what do they need data for and I think it's We can affect change in liberty and justice and equality Especially transparency and accountability and I think these are the main things that we can Effect change in and I think that's the key for like a new society So But who else can data help and I think lawyers journalists policymakers and advocates and activists can benefit from data and technology, but Maybe this is like a very hard Like they need the skills They need to gather the skills for them to have an effect on their causes and that's basically why I think this is very important so There I think also that intersections of data and liberty for instance or data and justice and data and equality can have very important impact in in privacy or surveillance or Or ending the war on drugs or the mass incarceration Phenomenon and then we can also through data and technology and Or at least shed light in biases Improve access to justice for a lot of people, especially the underrepresented people So I'm gonna Present the data for justice project, which is a collaboration Well, it's enabled by the Ford Foundation the ACLU of Massachusetts and Mozilla Foundation It started as my as the project of my fellowship like the open web fellowship that I was in 2015 and It became now Like my fellowship got extended at the ACLU and now as far as a Bergman Center fellow I'm trying to absorb Bergman's knowledge into this project So the data for justice project aims to empower empower lawyers and advocates to make fact-based and data-driven decisions and Do their work like advocacy? To improve access to justice in their communities by providing them and decision makers with answers to questions such as how isolation bias policing Race and poverty relate to each other What we're seeing here, and I'm gonna talk about that More in depth enough in the following minutes is this is Boston, Massachusetts What you're seeing is the green color is people of color Obviously the greener the color the more percentage of the higher the percentage of people of color live in that area and the red dots are Drug arrests So this is like only simple example of how data and data visualization in this case can show that basically The water on drugs is racially charged So I'm gonna talk about the process And how We try to implement this process at the ACLU and I think the first thing and it's The most important is how do you how do you get the data? We have here and Lambert which is One of the lawyers at the ACLU and she's very Like this is her topic like how do you get data? And I think freedom of information acts are the key But in many cases especially coming from the ACLU they get denied like so in one of these cases that I'm going to talk about we asked for police records and We waited for a year and a half and they wouldn't give it to us so we sued and also it's very important to Don't give up on getting the data because you want to change This problem, so we stood and we won and then After that I normalize the data and that's all that's my work like the The things involved are my work. I normalize it and process it and analyze it and visualize it But that's not it. It doesn't end there. I think we also have we have cave here, which is an advocate at the ACLU and their work is to socialize it and inform change and then we also have other departments that are that Their work is to make it last so if we use it as a flowchart It's okay. You identify a cause And if you have data well yay data If you don't have data, which is very common Do you know who who has data if you don't know? Well, you need to research and If you do know who has it and it's it's it's it open data If it's not well, you need to go through the legal channels, which are which are the ones I mentioned mentioned foias and lawsuits and Then you get the data and after you get the data This is my work. This is data science and you do you develop Measurements and indicators and you research and measure and you ask your question is this good is this good Is this good until it's not and when it's not you can design solution or or methodology to solve or improve this problem with community engagement and This is where Kate's work Starts and Yeah, and I think it's very important also from the beginning to involve communities in the solution. So it's more like a Co-designed solution because it's very easy for us That have the data and the skills to say oh, I'm able to solve the problem by myself and I'm going to reach out to communities with the solution and I Think that's not how it should work. Especially when you are talking about people Affected communities should be involved from the beginning of the and that's what the ACLU basically does but then if you want to change these things you need to also involve all the actors and Stakeholders, so you also need to co-design with them and you need to Create run workshops and do This co-designing for the solution I think it's very important that the government for instance, so it's also involved which is the hard part because in many ways You are saying to the government or to the public officials that their work is not done as good as it could be Right and especially in the case of the criminal justice system so You have The your workshops and do you have a solution? Do you have an idea that can be implemented? Do you have like practical? or empirical Solution that can be implemented and if you do well Yeah, you can relax a little bit and then make it last make it Through the legislative means or to other means to make it permanent I think it's also very important to think about that, but it's like a separate process and it takes longer and longer and longer so This is how it looks like all the this is the that I call it the data process and It's like an abbreviation and this is how I look the process of the data for justice project So I'm gonna speak about the projects like the particular projects we've been working on Here so we have three projects right now, which can be accessed the first one Is the do can case how many of you are familiar with the hand? Okay. Oh cool, so I'm gonna go inside it independent But we also have a stop on frisk data sets and the marijuana regulation That is coming in the ballot in November 8 So as you can see it's basically Work between the legal department and the advocacy department And they lies in the intersection of them of both. So the do can case And the do can was a chemist that working in a crime lab in Massachusetts and she was In charge of performing the drug tests for contraband or seizures she falsified Evidence and Thanks to data and thanks to the work that the ACLU we have come to the number of more than 24,000 cases so these are 24,000 people that didn't have access to justice that the that their rights were violated because the The I forgot the the work Okay, so yeah, the rights were violated because of this Action of the chemist. She was working at least for six years like six complete years and then Two more years that were half and half. So The impact is that a lot of people were in case incarcerated because of her and Some some say that there that there's no more people incarcerated because of her But still the impact is that some people were deported others have bad records that can also affect In other jurisdictions. So for instance, there's a case in Florida where a man is facing life in prison because He committed a crime or in Florida and the prosecutors Find out one of these cases and that will and that is the his third strike. So That that can happen Also One of the key of the key Things to know about this case is that prosecutors say that these are bad bad people that they were like plodding the the streets with drugs and in fact only six sixty two percent. I mean in fact 62% of the cases are possession only so it's Consumers instead of distributors that are affected by this So only thirty six thirty seven percent as our distribution. So we have some interesting tables So this table talks about is the division of the twenty four thousand and forty eight forty four hundred eighty three cases divided by county And also divided by adult cases and juvenile juvenile cases As you can see Essex and Sophoc are the top two clients of this this this lab and then This is the the distribution of The type of cases so sixty two percent are possession and distribution No distribution and thirty seven percent are other our distribution and one percent is other cases Which are like conspiracy and others that were not possessed possession or distribution Then We have this table that shows the percentage of cases with adverse disposition From 2013 and the percentage What percentage they represent in the total of the cases? So we have some counties that are that almost 30% of their drug cases were affected by this for instance It's I think it's then took it or what is this not Norfolk Norfolk Has 30% of their cases tainted and Sophoc has 32% of its cases Tainted that's a lot of people So yeah You can access these these Are screenshots from the affidavits I submitted that we that I worked with the lawyers at the ACLU So I have the links That I can send it to you because they are like very interesting and it's this only like a light Resume of this case So, yeah, I think that we can actually use data to change these 24,000 people Lives and I think it's very powerful Then another example of this work is and then I we can discuss and all that It's the yes and for which is the question for If you say yes to the to if you answer yes on the question for in the November 8 ballot You are validating the marijuana legalization well regulation and taxation in Massachusetts And the argument we are using and we are using data to prove it is that marijuana related police interactions are racially biased and Because they only happen mainly where people of color live and you can see that it's It's again that the same the same pattern Green color is people of color and the red dots are where arrests happened since August 2015 to date So, yeah, there is a I think a very clear pattern and I want to show you how this looks So I'm gonna Got no, sorry I'm gonna come here So this is an advocacy campaign that can be accessed by This website Yay Okay, so let me just marijuana policing destroys lives every week in Boston Even though Massachusetts overwhelmingly passed marijuana decriminalization in 2008 Right now an average of 13 people every week are arrested or harassed by the police because of marijuana Just One arrest can permanently change one's life for the worse causing barriers to employment schooling and housing the data shows the majority of police Interactions and arrests for marijuana offenses occur in neighborhoods where mostly people of color live Let's explore the data from the 2014 census and the data produced daily by the Boston Police Department 46% of Boston's population is white 25% of Boston's population is black and 29% of Boston's population are other people of color as you can see the city is racially segregated white people of color white people of color Now, let's look at where police harass and arrest people because of marijuana This is where marijuana incidents happened last week and This is where marijuana incidents happened since August 2015 Do you notice a pattern? white people of color white People of color we can help end racially biased policing of marijuana vote. Yes on four so I think that's This is the one of the examples of what data technology can help To do this type of arguments so Let's see How does this work and I'm going to talk about my tools Oh, I can't find a mouse Okay, I think it's here. No Oh You won't come. Okay. Here it is. Okay. Oh, did you get the URL it can be accessed by your phone and all that Share it so This is my work. These are my tools I do a lot of data cleaning and normalization and I think a lot about data structures I love like I consider myself like a systems architecture architect and also a data architect more than a that data scientists I do a lot of structures thinking and I do a lot of processing. I use tons of Python and PHP and SQL. I've been working on SQL for almost 17 years, so I know it very well and that enables me to leverage these skills to Actually make it very easy for us to it to output this process and Process data to other tools like a stata or are which are also things like I use and then also, I think one of the main skills is GIS which is the the things I use for and I think they are crucial for Translating all this data to where people live or where people is So that's why I love post keys. I think it's very important Tool and then to visualize it to tell a story. I think It's also very important and to do that I developed this framework, which I'm not going to talk about that much But I developed a data visualization framework that enables me to do that kind of pieces Like the one of marijuana legalization in very little time and with a lot of information being displayed and Analyzed so then its name it's ant which he which stands for augmented narrative toolkit and it in it be it's built on Already developed tools like the tree and a score magic and jQuery and all these other tools I don't want to reinvent the wheel. I just want to implement road to make it easier for people to To use these very complex tools like the tree so my aim by developing ant is for you to be able sorry for you to be able to Put a map or a chart in a web page as easily as you will than image Which is like an a standard HTML Tag so my theory behind that is that if you enable people to use HTML To do data visualization HTML and CSS. I think that we are going to start to see a data-driven Internet and I think that's that can be very very cool But then what the visualizations are cool But I think the most important Thing it's the average we want to have impact on people's lives I mean the technical challenge is there and it's kind of cool to solve it but I think that if The work I've been doing doesn't have an impact on people's lives I think I wouldn't be happy about it. Like I will be only half happy So I think outreach to communities outreach to the authorities or the lawmakers. It's very important too and so for instance, there's a case where One of these visualizations got the attention of the Urbano project which is a youth of color workgroup in Jamaica Plain and what they asked the their fellows is to create this To do something with it with this data and what the kids did. I think was like a Amazing thing was a they painted the map that I just showed you In a mural in Jamaica Plain and they marched with that mural Through all these neighborhoods and I think It's very low-tech, but I think That's like the that completes a cycle of data Because that specific those data points that those specific data points went from The asking for the data in the foyer then suing for them for for it Then it got processed and we reach out to communities and the communities did something that then impact in in other types of stakeholders, and I think When I saw that I cried I was like, oh, this is this is how things should be working. It's not about the technical challenge. So My conclusions are that data and tech doon don't do anything by themselves people do Data science is very cool. Very very cool. It's a technical challenge But it's only a part of of a team with a common goal I wouldn't be standing standing here if it were not for my co-workers a co-workers at the ACLU that are kick-ass And also the intersections are very important exploring law and data it's it's very cool exploring data and Inequity is also very important. So yeah data and tech in the hands of the public interest organizations Working to reduce inequality can have an impact on people's lives Thank you So if anyone has questions Okay Okay, thanks. Hi, I was just saying that I love this project and all the work that you're doing with your colleagues and My question is about in the beginning of the presentation in your flowchart You talked to there's sort of a parenthetical statement where you said you could also involve the community that's most affected earlier in the process For example when in data gathering or in identifying the data sets that are needed And I know that Urbano project that you just referenced they did a project like that where? They were looking at transportation Inequality in Boston and they had young youth of color from Boston keep journals of their transit times like to school to work that type of thing and they created visualizations of Transit inequality by race and geography in Boston. So I wonder if you could talk a little bit more about Yeah, what are what are the? What are the challenges that are keeping more? Data scientists or people who are doing the type of work that you are from engaging Communities that are most impacted at different points in the process Why does it happen so infrequently because I think it's hard It's it's expensive and Yeah, I don't know if it were not for me being enough in a paid fellowship I wouldn't be able to do it because raising the money to compete with Facebook for the ACLU is a challenge and Then Ritching out is so it's a full-time job. Like it's it's not very easily done and you need to have like a strict credibility And all that you need to have connections to the communities and I think that only through the collaboration with With organizations you can do it like community organizing for reals. I think that's that's the main challenge So data scientists are very Like technically skilled, but they it's they're very far from in I'm generalizing of course what from community organizing Great great project and the visualizations are compelling, of course I wanted to so a lot of these data are very personal and private there is data about race and criminal convictions and location and I'm wondering what's done in terms of Privacy in the data sharing process. Do you get the raw data? Do you get like? to just perform queries and what's the privacy Cover for this well So in a look and case I had access to private information like personal information that that includes Names and social security numbers of people Like that were charged in these cases These data seem pounded and it's like I use security measurements to keep it that way and That's one case the other maps are open data So there they've been through a process of the anonymization and So for instance, if I showed you This So these are not where exactly happened. These are like agreed So agreed that this I think half a mile So if an incident happened there, it goes it goes to that point that enables us to to remove the identification layer so people or for instance So so people don't can't say where something in particular happened so that they don't have the repercussions of being the anonymized Pella I my name is Kate Crockford. I work with Pella at the ACLU I just wanted to actually to Pallas horn a little bit more than you did To tell you all the significance of having a data scientist on staff for the legal department in the midst of the bridgeman Litigation so the bridgeman litigation is the drug lab case We got the disposition data through that litigation as Palla said some of it was impounded We filed a motion asking the court to unseal the non personally identifiable information aspects of that data which they did and then Palla was able to perform this really important analysis which Revealed things like the fact that 62% of these cases were possession only which nobody knew before that So that was reported in the press as a major finding We that was like a major part of our brief one of our briefs in the case and then You know that one in three cases in Suffolk County was tainted by this scandal I mean these are these are really important pieces of our legal arguments in The bridgeman litigation that we would not have been able to make had we not had power on our staff So I just want that to be very clear and also I would just like to tell you I don't know if you know this But the prosecutors filed a motion last week that pretty much dis is Palla really hard and says Which I think is great because it means you're doing the right thing people are powerful people hate you already so fantastic They said They said things they effectively said I mean if you wanted the bridgeman litigation is really insane I encourage you to read about it because it'll put your head in a spin Cycle so one of the things they said about Palla is that It was not the intention of the court when they unsealed this information for anyone to actually look at it They really said that the prosecutor said that in a brief to the court Yes, the court unsealed this information But the purpose of unsealing it wasn't so people could understand what was going on Okay, and we're really upset that this meddling Palla via real person you bothered to look at it So just so you know she's really making an impact. It's really valuable from the ACLU to have Scientist working on information like this. Thank you Is there a is there a page on the web anywhere where you? Link to all the tools that you use to do this So that other people can do the same thing and follow your methodology. Yeah, we have a github page I mean all this except for the Durkan case, which is like a court case and I'm going litigation all other things are public in github Yep, so you'd have Data for justice Yeah, so you Will be able to see the code for these which is in the give this a github page, so You will be able to see this The methodology that I need to update and this is like this describes the The process to get that from the census data to all the police data and all that so and there are other pricks so Yes we know what happened to the person who falsified the Data, but the the question is what happened to the people were in jail So some are Maybe I'm can answer that question in particular. I'm sorry to put you on the The response of the legal system was to recognize immediately that the people who were serving time At that moment when this was discovered had to be have an opportunity to challenge their convictions and get out and in Pretty short order the court system mobilized and had Special magistrates appointed special hearings all of this by the way to follow up on Cade's point was challenged by the Commonwealth Oh, you can't vest those retired judges with judicial powers to admit people to bail or even to have hearings Where they make recommendations to a real superior court judge about releasing people on bail but the Litigation that's going on now and who knows I mean I think probably the court system did and a good First cut job and figuring out who was still incarcerated and doing something about it. The problem was as Paula has described That Then a certain amount of inertia lassitude Set in what about the 24,000 other people? What what are the remedies going to be for them? Should we do something or The Commonwealth seems to say, you know It's just so formidable You know finding these people What, you know, gee it's so overwhelming. You know, we just can't handle that So the point of the bridgeman litigation bolstered by what Paula has been able to demonstrate is You know, we have to do something about it the burden of these Tainted convictions shouldn't fall on the people who got them. It should fall on the people who went about Prosecuting them and then imposing them. So that's the issue that's going to be argued in the court next week Is I mean that's semi an answer. I mean it divides the world of duke and victims into People who were incarcerated where the system sort of responded But not fully and people that the system At least with, you know, Commonwealth versus John Smith the Commonwealth was kind of willing to let those go where the People in the bridgeman case Our organization the public defenders have said, uh-huh. Uh-huh. You got to pay attention to them too. Yeah I've been conducting hosting a discussion in the context of a Harvard ex-mooc called jewelry X Which began discussion just this week of the issue of yes on question for and the most provocative Hello Yes, the most provocative question has been the The way a way how to think about the fact that The leadership of the Commonwealth in the person of the governor the Attorney General and the mayor of Boston have been absolutely opposed to the yes on for and The question posed that was provocative is is there a relationship between The uniformity of the opposition of law enforcement Emphasized in this morning's paper by the support of the prosecutors for the opposition police opposition to yes on for Is there a relationship between cannabis reform and black lives matter and You've answered that question Your your work is such a beautiful Demonstration of it. Thank you that it is remarkable That black lives matter as a movement hasn't cottoned on To cannabis reform as a sister issue. I mean look here. We are We're not a black lives matter group and yet we can see it absolutely clearly It's a curiosity. It's a But thank you very much for the work you've done. It's extraordinary. Thank you. Thank you well I've been reading Kathy O'Neill's new book weapons of math destruction Which is a terrific look at how algorithms are impacting Vulnerable communities around the country. Oh, sorry. So it's a book called weapons of math destruction It's written by this woman Kathy O'Neill Who was at D Shaw and as an implied math person? And I was thinking about that and about your Brandeis quote from the beginning and I'm wondering if you can talk about The origins of your passion for social justice as a data scientist and what we can do in the data science community to get Data scientists excited about this. Do we need some sort of formalized training the way we do pro bono for law students? Okay, so To answer that question I will have to tell you like my background, which is I'm Mexican 32 years old and when I was 15 I started working well I learned to code when I was 12 and I only was able to do that because I Had access to the internet and I started talking to people that were Coincidentally Also developing the chats we were using So that I don't know that picked my curiosity So that's how I learned about Linux back in 1997 Then I learned the about C the language So fast forward three years. I started working as a web developer in Mexico when I was 15 and then Fast-forward four years. I was the sysadmin for Mexico's president's office And I was in charge of securing the web servers and enable enabling Like a development platform for the Mexico's president's office So I started reflecting on what had enabled me to do that because at that moment I recognized that I didn't have like peers my age Doing the work I was doing So The answer was open source and openness So I think that and that's when I decided that Through openness and open source. I could actually enable people more people to do the work I'm doing and now fast forward 12 years and I'm doing that. I'm I've been working in governments. I've been working with sexual and reproductive Organizations freedom of expression organizations And all this type of work are only trying to demonstrate that openness and open source can actually change people's lives and now I think my My goal is to inspire inspire other people to do just that So, yeah, I think that's That's it. I Thank you for the Ballot question on for now. It does bring up a point of You know what in data analysis like this? There's the legal aspect that you dealt with and and certainly You know, that is very apparent now. What is the what's the social impact on this type of data analysis where? cannabis reform is One part is the legal aspect of it. The other part is the effect on the community as a whole What kind of data exists for gateway drugs? In in other communities where it has been decriminalized That's what The social impact would be is there any kind of work along those lines because the the ballot question and I'm speaking as a as a Loan voter out here is flawed because it deals only with the criminalization, but it is not It's it's actually about the long-term social impact that can happen in this community What has what is the data that exists elsewhere? Yeah, do you want to answer that okay? Sure, I can try I Think I hear you saying that if we tax and regulate marijuana more people will smoke marijuana. Is that what you're saying? asking about the data itself must exist not only in the United States, but in in neighboring countries where The decriminalization has taken place. So that's what I'm asking about is that the data analysis has a social impact Right, I guess I'm asking what kind of data Data about use or data about to use yeah, and whether whether in in terms of the Where the cannabis is a drug that One shouldn't be bothered about you know as opposed to to heroin or crack or something like that Well, this is a little field of the subject I think but just very quickly there is data from Colorado that shows that Basically marijuana use rates don't go up significantly at all after legalization Also, we have past decriminalization in Massachusetts So it's already not a crime to possess under an ounce of marijuana for personal consumption And we haven't seen measurable increases in use since then what we have seen however is a Jurassic decrease in the number of people who are arrested for marijuana possession in the state So clearly as Paula's data shows The problem is not over and unfortunately the reality in Massachusetts what Paula's map shows very clearly I think is that Really only people of color are facing any kind of law enforcement interaction because of marijuana today in Massachusetts So Decrim had the effect of making it possible for me to smoke weed and not have to deal with the police But not for my black and brown neighbors So that's really the the issue that this map brings to the fore and and so if you're uncomfortable with those racial disparities in Marijuana, you should vote yes, it's my view Yep, yes, thanks for your work. It's fantastic I have two comments one black lives matter has looked at this issue, but they're not narrowing it to cannabis use only number two The racialized and gendered aspects of this are really troubling People are not looking what's happening to black women Okay, they get criminalized at a higher rate even compared to black men So it would be helpful to take a look at that population segment of the population as well And finally has anybody looked at environmental justice there are environmental justice communities right here in Boston that no one Has done anything about from decades. Thanks Thank you. Thank you for your talk. Um, my question is about How do So you're doing work and I know you're doing good work and you're reputable and you're working with a very reputable organization So the graphs and charts I see that you're producing. I can be Personally trust as reliable I'm wondering if there's any as we're getting into an era where we're using a lot more sort of visualization data if there's any kind of ways to Evaluate whether something is Valid as opposed to propaganda say even if it looks valid and I mean we can just see like how people think people can spin things in all sorts of ways and a lot of us might maybe can't necessarily look at the code and Or anything so I'm wondering do you have any ideas or are there any organizations that will evaluate sort of the data That's being put out in visualization form to tell you if it's Is trustworthy that's a great question and I don't know how to answer and yeah, we are Basically appealing to our reputations, you know, like yeah, so That's also why the code is there, but I recognize that not all the people can read the code and Basically, of course in the Brisbane litigation one of the arguments of the DA's Was that it was not peer reviewed like the my affidavit wasn't And yeah, it is not right so But that doesn't mean that the But to Contrast that I think what I'm doing is not an analysis Right, like, you know, I'm not trying to I'm not using AI or Deep learning or anything like that. I'm trying just to describe things and I try to keep it Like like that. It's like this table is brown and the computer is gray And if you put them together, it's a you know stuff like that. It's just description That way, I think I can convey the message very easily without trying to To like involve more More thinking or more confusion. So, yeah, I don't know. It's it's so complex. Yeah, but I think thanks for asking Thank you so much. This has been amazing I wonder it's clear that there's still a great deal of work Valibable work to be done with this data set and I look forward to hearing more especially in light of next week's Court proceedings, but I'm wondering if you've identified like, what's the next big data set you're going to go to work on that might produce Marvel's real also. I'm sure there are many out there. Yeah, one of them Like I developed tools to explore data one of them is like a sunburst like a peak pie chart but and they like In last in how do you say like an expanded pie chart, you know, like a sunburst That's the name of it. And what I use it for I can show you is For instance with the payroll of Boston's Boston's City employees, yeah Table So we have I already imported the data sets This is the payroll and these are their columns. We have the name of the employee the department It's their title and then we have all these Income categories and the zip code So if we want to see divided by department We can see that Boston police department and Boston fire department have a big chunk chunk of the payroll like the employees. These are only counting and counting employees But if we if you want to compare it to Yeah, in terms of money, let's see Total no way to restart because Okay So we have this and then if we want to do total earnings and we say Some and it's money and we do this and now we have to this is the total earnings and This is the size of the department. So you can see that Like in proportion BPD is Getting more money, but if you want to divide it by I don't know Title it can get very complex, but it's a lot of data You start to explore these type of things and now let's focus on Oh Is that for example Boston police officers who work in in West Roxbury get way more over time Than police in other parts of the city, which is so weird It's like what's going on in West Roxbury that all these cops have like huge Hugely greater percentage of overtime than police elsewhere. So that's the kind of you you know unique insight that these tools allow you to see that You wouldn't be able to see elsewhere. No police are people who work there. This is open data Yeah, this is only for the seat of Boston open data. Yeah, so now we have this which is Boston Police Department only and then by title which is police officer and then by zip code so this enables you to see that the thing that Kate was saying which is that in this in this zip codes like the police officers make significantly more money than In other zip codes Yeah What I mentioned before like even before thinking About data is the the mission like that. I consider this being a mission. Oh, sorry mission driven Data advocacy so that that's how I see it. It's like we have a mission which is prove that the Interact in the police interactions are racially biased And how do you which data do you need? So you start to ask these questions through research? And in many times you will find that there are government Departments that have data a lot of data, but And this is one problem I faced in my one one of my previous jobs, which was In Mexico City's government that we had a lot of data But there was no impact of that data because there was no translation From that. I mean first they don't know They don't know who is going to use it And second they don't actually do the outreach for people to use it So it's it's like a I don't know like a nega chicken problem. I think but if you put a mission on it and You can start to make all these diagnosis of oh, I need this data set and And then I need to ask this person for what this column means and then I need to Do that, but yeah, it's it's very specific and very mission specific things. Yeah, it's there's no explorer and there's no like It's complex Thanks so much for the talk I'm a journalist in South Africa and a lot of my work is bringing sort of information Complicated information to sort of communities. I was wondering what was your interaction like with journalists? Did you bring them on early? Did you work with them in that way or was it a case of it's finished and then handing it over? What was that sort of process like we haven't had that much interaction with journalists yet? Because I mean we It's I love to to have more more Contact with them, but it hasn't happened and I really don't know the ways to do it and also that The little contact we've had this like they won't publish the maps. They are only going to write about it You know like especially here that it's my understanding that understanding that Boston's Media is very specific in its in its actions and how how they handle cases Did they feel that they want to have had control over the whole process to publish it like it's almost like they don't want to I don't know I I mean it's also through that I'm working on an advocacy organization Right, so it's it might be also that that they want to replicate my findings and they But they need the skills to do it. So Yeah, it's it's like a Catch-22 what I love journalism and I think law and journalists are that the like the the key of our society, but yeah That's great. I think we'll probably leave it there. Do anything power for this