 Hi, everyone. Thank you for joining us today. We're going to get started in a couple of moments. I just want to acknowledge you all in the room, though, first, and also our folks in the virtual audience, too. Thank you for coming in today. Welcome to our Brinkline Center. If you want to stay involved with future comings and goings at the center, please head to our website for brinklinecenter.org slash get involved to hear more about our wonderful events and wonderful line of speakers. Without further ado, I will now hand it off to our actual speakers of the day. Thank you all so much for joining us. Hello, everyone. Welcome. My name is Adam Holland. I'm the project manager for the Brinkline Center's Luma database. I'm thrilled to be here moderating what I expect will be a fascinating and fruitful discussion. Before we begin, I would just like to alert everyone that this conversation is being recorded, but that the recording is facing us. None of your faces are going to be on the recording. And so you don't need to worry about that. However, if you choose to participate by asking a question on the mic, your voice will become part of the recording for the event. So we anticipate having a fascinating conversation here. And then we'll spend the latter portion of the session opening it up to Q&A. So I hope that we'll have a lively discussion on that front as well. Without further ado, I'm pleased to be able to introduce to our guests. To my immediate right is Felix Rita, who is a former member representative to the European Parliament and is currently a project manager at GFF. I'm going to give it a try and you guys can laugh at me. I just want to show off the right height record, the Society for Civil Rights, which is a German organization that defends fundamental and human rights through legal means. And we are also joined to his right by Malta Spitz, who is the co-founder and secretary general of GFF. And while they're certainly going to tell you more about what they're about. I'd like to bring to your attention the brand new European legislation, the Digital Services Act. Most specifically, its provisions involving researcher access to data and the affordances and opportunities that that might provide for strategic litigation in the public interest. And so, I'll turn it over to you Malta. Thanks a lot. Just say a couple of words on our NGO work and afterwards Felix will explain a little bit extra on our PSA work too. So, GFF is an NGO who is really focusing on strategic litigation. It's not really an issue about it, not only in the digital rights field, but it's like in the fundamental rights all over, but as a specific, I would say like a specific focus on digital rights. At the same time, we also do cases on equal pay on anti discrimination on state surveillance also old school type of the state surveillance, not only a digitalized state surveillance, and also examples on cases on refugee rights. And now for the last two to three years, we have opened up more also challenging companies to so our work is mostly against is often against state actors, but now you're also moving forward to go after companies, we already had a case against metal, for example, which we won in the first instance and it's a DSA I think we will start even more strategically litigation against large online platforms and Felix will explain a little bit extra on the ideas of also on these new possibilities. Should I jump right in. Yes. So, I was a Bertman client fellow after I had the parliament and I was here in 2019 2020. And at the time, I feel the European Union was looking at platform regulation very differently so I was doing a lot of work from the users of online platforms, especially in the copyright field also where we had a lot of interactions back then. At the time, there were issues around that content ID, for example, inadvertently blocking legal forms of speech, and we had you legislation that would really compel companies to block all kinds of legal issues and quite often freedom of expression issues were kind of left at the wayside because platforms would be kind of compelled to use relatively simplistic solutions to quantum moderation that would adversely affect the rights of users. So, with the Digital Services Act, which was adopted about a year ago, and is about to fully become applicable next year. I feel that the the tides have really changed because the Digital Services Act looks at users not so much as potential criminals or just consumers of what is going on in the online ecosystem but really as citizens who have rights and it's kind of taking a much fundamental rights friendly approach to this so I put only three slides together because I don't want to bore you with a presentation just to explain a few basic concepts if you have not followed the Digital Services Act before so this is kind of a simplified point of view what kinds of online intermediaries are regulated by the Digital Services Act. So it's a system of kind of increasingly strong obligations, but has very basic obligations for any kind of online intermediary so that's not just social media companies and things like that but also for example your internet service providers, DNS resolvers and so on and those basic obligations for intermediaries are very light. It's primarily like if you have customers in the EU, you have to have a legal representative in the EU you have to be contactable for regulators and things like that. Increasingly more burdensome or more strict obligations as we go into hosting providers so for example, up until now the EU did not have a notice and action system laid down in law. So it was really quite a lot of companies were basically following the US Digital Millennium Copyright Act in the EU for copyright matters and also for non copyright matters that was kind of the sort of in official framework of how to deal with potentially illegal content. And so now with the Digital Services Act for the first time there is a notice and action system laid down in EU law that applies to all hosting providers. A subset of hosting providers are what the Digital Services Act conceives off as platforms so these are primarily kind of public facing hosting providers that organize the content that users upload so for example, a simple Weckles thing service would not be considered an online platform, but something like Facebook or Wikipedia certainly would. And those platforms have increased responsibilities when it comes to content moderation so for example they have to have a complaint system if something has been removed or a decision has been taken. So people have stricter transparency obligations when it comes to how their recommender systems work and things like that. And finally, you have the most far reaching obligations for the so called VLOPS so VLOPS stands for very large online platform it's not very. Very imaginative term, but these are platforms that reach 10% of the EU population so something like 45 million active users, and they have to actively engage in a risk assessment of certain systemic risks that are associated with online platforms in terms of risks in terms of dissemination of illegal content but also risk in terms of the fundamental rights of users, risk to public health, electoral discourse, and things like that. Basically, those VLOPS are also the ones that will have the most obligations when it comes to requesting access for research purposes if there are research teams that want to look into those systemic lists and that's primarily also what we want to talk about today because in terms of the society for civil rights, we are really interested in testing these provisions in court and seeing what we can get out of them in terms of improving the respect for users fundamental rights in the online ecosystem. But we are an organization of lawyers, we're not an organization of researchers or data scientists. And so we want to get the word out about these new opportunities to. Hopefully also encourage researchers to think about how this might be useful for your work. This is a great slide, and one of the things I think may not be immediately obvious to someone who's just coming to the DSA and its researcher provisions, its requirements for the topic is how hotly contested. These tiers are. So, if you haven't been paying attention to this, what about six weeks ago was the first moment wherein online providers had to self certify to the EU whether or not they were in fact a VLOPS, because if you are the calendar for your requirements is significantly accelerated. This is a VLOPS meta is a VLOPS Google is a VLOPS, but a little further down the list there was some, to me at least extremely intriguing. Yes knows, most notably that Spotify, which I think simultaneously said, they have over 120 million people in the EU, but was not a VLOPS because some very arcane reasons. I wonder if maybe you can talk a little bit more about that and why the requirements for VLOPS are so powerful for this is engaged citizenry you've described but maybe not quite so desirable for the platforms themselves. Sure. So, what we've put on this slide are the ones that have self certified as a VLOPS. So those are the ones that, you know, don't need to be brought it by the regulators but are rather kind of saying at the out there. Yes, we do consider ourselves to be a VLOPS and there you kind of see you have a few search engines which is also bizarre I mean I have to admit the last slide was simplified because search engines were added very late in the legislative process to those obligations for very large online platforms even though they're not platform they're not hosting providers so there's a little bit of a lack of congruency when you look at how it was designed at the outset. In the end, search engines do have to comply with those risk assessment obligations and so on, even though they're not hosting providers, but yeah so you have a few search engines for example, you have some e-commerce platforms and then you have a whole bunch of social media companies and then Wikipedia is the odd one out in a way that it's not a company, it's the only nonprofit on the list and also has a very different approach to quantum moderation than any of those companies. So those are the ones that are for sure on the list and then there are a lot of question marks so I would say in terms of Spotify, it's something that could be challenged by the European regulators so the European Commission has the power to, to, yeah kind of ask additional information or also question this self designation and the other obvious omission from this are the adult entertainment platforms that all claim to have fewer than 45 million users in the EU, which is also open to debate, I would say. There are others that are being asked about like Airbnb for example where I find it easier to believe that perhaps you know not quite as many people travel and rental hotels as we might think, but yeah those are the ones that we know for sure that are self assigning as we also at this point. Next one. I would just maybe quickly go back to the first one because so when we talk about what researchers can get out of the digital services act I think it's worth noting that not all of the requirements for transparency are limited to those platforms. So if you are researching one of those platforms that was on the last slide, great, there will be quite a lot of opportunities to to research them, but there's also some interesting stuff that applies to all platforms and products also. There's more about that because there is a mandatory database in the digital services act for the content moderation decisions that was very much inspired by the project. And yeah but unlike Newman it's going to be mandatory for all platforms so quite interesting maybe we can also discuss from your experience like what kind of research Newman has facilitated as a voluntary database of content and maybe what you see also might be interesting research questions that can be, yeah in the future hopefully answer to the digital service. Sure. So, I see like points out the DSA and articles 17 I believe describes the mandatory database that must be built and hasn't been built yet we're very interested in what what ends up being built. It's being populated, essentially by the content moderation decisions that are made by all these platforms, and the VLOPs have to go into a little bit more detail but all platforms that are making content moderation decisions are required to contribute to the database in some way, and quite similarly to what Felix mentioned about the EU, essentially using a notice and takedown regime although it was never instantiated in law sort of flipped that a bit. The United States Digital Millennium Copyright Act was the first legislation to put into law a notice and takedown regime it's from that law that we get the concept of notice and takedown. But intriguingly, there's no piece of that law that mandates transparency. So I run the Lumen database which is a Berkman Klein Center project and we're a database that collects copies of the notices that are sent as part of notice and takedown. And we have about 25 million at this point, which we gathered over a period of 20 years. And every single one of those was shared with Lumen voluntarily by a platform out of a greater interest in transparency, but they could stop. They could take them back, it's completely voluntary and so one of the things that's most incredible about the DSA is that this database is mandatory and it will be universal participation and so, as Felix points out, the possibilities for useful research are almost to me at least incomprehensible because we know we've been able to do with just the voluntary data that's within Lumen and the idea of expanding that by several orders of magnitude is just astonishing. A few brief examples, is that appropriate at this juncture? Yeah. So, Lumen was started to collect copyright takedown notices, and it was exclusively that for many years and then as that mechanism spread into the zeitgeist and people began to realize, well hey, I can take things down. I don't really know why or I may not have the legal pieces in place, but it's worth to try. The database has expanded to include takedowns related to defamation, private information, local laws. So if you're one country in particular is something illegal to be online, someone in that country might send Google or Twitter or whomever a takedown notice. It's for them to respond either globally or geolocationally. I mean, if Ross come nods or which is the Russian regulatory agency says Google, you're linking to material that explains how to use drugs. It's illegal into Russian law. Take it down. They may take it down globally that they could take it down, take it down only in Russia. And stuff like that. But what we've also started to see with respect to this expansion of the idea of noticing takedown and also the type is what I can only describe as weaponization of takedowns. So for example, Walter if you write a book, and Felix thinks it's the greatest book in the world and everybody should get to read it for free. He copies it and puts it on their website. You could completely legally send a copyright claim to his hosting provider or to Google or any other search engine to de-list it. My copyrights been infringed. Take it down. However, what we're seeing is someone who doesn't have the copyright. Asserting they have the copyright in order to achieve the removal of material that is hardly adverse to their interests in some way. Could be critical. Could be reputation damaging their opinion. And so they'll send takedown notices or other notices to minimize their online presence. And so we've had researchers both in the past and currently working with the limit database rather elaborate schemes to do this, the scale of which I continue to be amazed by. So I'll just briefly name a few. You can always reach out to Lumen and we're happy to go into more detail. I'll give you a researcher access yourself. But just very briefly. Right now, our Lumen research fellow Shreya Tabari is working on a large scale scheme to weaponize DMCA notices copyright notices to control reputation online and so she's identified a subset of approximately 100,000 notices, all of which can be traced to the same group of senders and from there probably to one individual who's paying this group of senders. And they use what's called the backdated article technique, which is, oh no. A newspaper has written an article pointing out that I'm violating you law and maybe even have assassinated one of my rivals. I can't let that show up online. If someone searches for my name. So I pay my organization to create a completely fictitious website that looks like another newspaper. I copy that article onto our website, and then I just changed the date so that it appears to have been published a week earlier than the original. I can copy right notice to Google, or whomever and say the original has copied my big article, take it down. And because of the incentives of the MCA. It's usually taken down. It's very easy to realize that this is a fake. If you're a human being doing the detective work, but given that probably hundreds of thousands of these notices are sent every day around the world. So very few of these organizations have the resources to do the human detective work. And so that's one of the things that Lumen makes possible by aggregating these. And so we've found this out we've written it up we've received bizarre threats from Swiss lawyers to drag Lumen into Latvian criminal court. It was funny. We're publishing this work. So there's that out there people are weaponizing existing legal structures to control the reputation online, closer to home and perhaps even more eyebrow raising Lee. We have a US law professor who's explicitly interested in court orders. And something came before a judge in a US court. There was a proceeding and the court issued an official order, or since it's Lumen it's in order to take something down off the internet. And for most platforms that's the gold standard. It's not just a DMC request it's not just a kind letter it's a court it's a proceeding you have to honor the law. To my, even years later continued surprise. This research found that as many as 10% of a sample of several hundred court orders were fraudulent in some way. Some of them are explicit forgeries and you're thinking what's a forged court order look like. Well, sometimes it's literally someone photo shops. The URL out of a real court order puts their URL in and then sends that to Google. Again, pretty easy to figure out if you're looking for a problem, but it's like oh yeah this is cool. You're not going to notice. That's on the one end of the spectrum. You can find that particular instance is what he's articles written about it. And the perpetrator confess later and I believe this quote is a paid to some teenage kid next door $500 to do this for me and it's all down from online. On the other end of the spectrum, there's a California company which has since gone bankrupt and its founders are on the run from the FBI that had a quite elaborate business model where and they would fake a power of attorney, fake a defendant for a John Doe. They would draft a stipulated order of settlement, take it to an out of state judge and an out of state lawyer, who everything looks good, they sign off on it and then have a real court order real lawyer real court real judge, found it on road, and then take that back to Google, and this only came to light through research on women led to a federal, excuse me not a federal Texas state prosecution and the dissolution of this company. So those are just two examples. This is the kind of thing women exist to facilitate this kind of research and this kind of social impact. And that's only with voluntary disclosures on the part of the 15 to 20 companies that share takedown notices with women so I asked you to imagine if every single company in these categories is submitting a notice and content moderation decisions. I would hope that the environment for deliberate misuse and abuse would be much much less friendly. But I also suspect that we are going to learn that there's a much higher rate of incidents of misuse and abuse and anyone has previously imagined. On the more positive side, it's going to make possible public interest to choose your litigation with user interest somewhat. Right. So maybe we should talk a bit about the research data access request because one thing that we have found is that if there is a takedown notice I guess the platform has an interest in transparency at least some of them know which is why they don't necessarily participate in women, but of course there's probably less of an incentive for the platforms to share information about their own content moderation that they do. You know, algorithmically their own recommender systems and so on. And so beyond kind of these clear cut cases of something has been blocked or has been kept online. Research data access provisions which only apply to those large platforms allow request researchers to request additional data that is necessary for public interest research. So, basically, this is something that anybody who is affiliated with a research organization can try to get access to. So it's a complicated process which I will try to briefly explain, but I think it's useful if you're researching any topic that is related to risks on platform so the systemic risks are listed in the digital services act but they're quite so they include things like dissemination of illegal content. They also include risks to fundamental rights of users and they're based based on the European Charter of Fundamental Rights. It's a pretty broad spectrum from speech issues to equality issues to social rights. The DSA also explicitly lists electoral discourse or risk to electoral discourse things like this information in the run up to an election. It lists public health which was probably very much inspired by the platform issues surrounding the pandemic. It also lists this gender based violence as a potential systemic risks. So if you are researching in any of those areas you can try to get vetted by the European regulator as a researcher and this regulator is then going to make a request for additional data access to the platform on your behalf. So the way that this works is you can either go to the regulator of the country of establishment and this is probably going to be from sometime next year. Most of those platforms that we've shown are registered in Ireland. So usually it's going to be the Irish digital service coordinator, but you can also if you are affiliated with a new research institution. You can go to the digital services coordinator of that country for help. So, what we've seen in the past with the data protection regulation was that the Irish regulators were somewhat overwhelmed and overstaffed, and it's kind of debatable to what extent that was just a question of public institutions having to go after very large companies with nearly unlimited resources, but the digital services act has tried to learn from that process and has given more power of enforcement on the one side of the European Commission, and also to the other regulators of other countries and also to a certain extent to civil society, who can bring claims on behalf of different interest groups, which is also where we come in. So, you try to get vetted which means you have to describe how your research relates to those systemic risks. What your research question is, and what data you need from one of those very large online platforms. So the digital services coordinator, which is the regulator then makes a data access request to the platform, and the platform can either comply with that request or they have to justify why they cannot and the reasons for refusing access are fairly limited because you don't have the data. So the platforms do not have to provide data that they don't have. And the other, which is a bit more problematic is that disclosing the data would violate their trade secret protection which can of course be an incredibly broad reason for refusing data access but in those cases the platforms have to make a counter proposal. And to say, well, this is what we can provide to to answer this research question, and ultimately the decision that they're going to be up to the regulator whether to accept that or not, if the regulator refuses the data access. The researchers could try to sue the regulator or if they grant access the platform to try to challenge that decision court so we do expect, especially around this trade secrets question quite a lot of litigation and where there we want to also contribute to a solution that does not, you know, make this tool useless in practice. What the researchers also have to do is to to provide a concept for how to account for data protection and data security, and they should already do this as part of their initial request so that the regulator can evaluate whether they are capable of keeping the data that they're requesting to be safe, especially if it's sensitive personal data. And in the proposal there is also a room for an independent third party body which would act kind of as a clearinghouse between the platforms and the researchers and also help in this process. So, what we want to kind of explore, maybe in the remainder of the hour is what kind of research questions might be suitable, keeping those systemic risks in mind and also keeping the companies in mind that the Spotify is very large online platform so maybe we can put them back on just yeah to to aid as a visual reminder so I come very much from the copyright field so I'm personally quite interested for example in how social media companies like YouTube, which I think doesn't contribute to Lumen, like how they have dealt with this because like under EU law at least platforms like YouTube are required to use upload filters to block copyright infringements but only if those filters make sure that legal forms of expression like quotations for example actually do go online. And it's a big question like how they actually do that in practice and whether they do that because content that never gets uploaded in the first place you don't know that it has been blocked. So there I think it's a lot of room for using on the one hand this database of content moderation decisions but also for researchers potentially to ask follow up questions, but I could imagine that beyond the copyright field in other areas that would also be a lot of potential use cases also looking at e-commerce platforms and things like that. So with that in mind, shall we turn it over to the audience to see what their particular initial interest might be? This is really Tara and Cognita for all of us. So we turn it over to you. What opportunities jump to mind or what would you like to see happen? It's still very much a work in progress. As I mentioned the database does not yet exist. It's not Lumen because it's really the only similar database in the world right now that's aggregating this kind of data we've been in conversation with people at the EU commission about ways in which they might usefully structure their own database, ways in which they might vet researchers and the possibility of what I think of as tiered access to sensitive data. I mean because it's hard to think of data that no one should research. If it's that sensitive, it's probably pretty important. But what's on your minds? I turn it to the audience. Yes, please. There's a microphone going around so if you don't mind waiting until you have it. So speak twice. Yeah. And he hoped PhD candidate over at MIT. Part of my research I do on on YouTube, and here I would like to bring on something that I see as a limitation but I'm we're most curious about how to ask the question that that's around digital access. A lot of the potential system of risks or harms. I would place as not only relating to the platform and not only relating to the user but in the interaction of users and platform, for example, all sorts of questions around radicalization polarization to recommendation systems which have been in prose a lot talked about a very little empirical support which like actually tests whether this happens. And there's always the problem that if you get some data from a platform, then there is the innocuous explanation. Okay, these people were just in a certain environment where they would have anyways, radicalize it's not because of the platform. So you would like to somehow establish some causality, and one way to establish causality in platforms and that's done all the time is AB tests. So you're testing one time this one time this and you look okay that does this change something. And in my experience of talking to people in human subjects boards and so on. I think it's associated that we are being tested on all the time by platforms. And in the data access provision in the DSA right now I think it's just saying data. But it doesn't seem to me that we are going to be in a world where we're definitely not going to be in a world where I can come as a researcher and tell YouTube okay please run an AB test for me. I still need to work around that and get a browser extension get the legal help with that. Because my participants are violating terms of service from YouTube. This still seems to exist. Is there a way in which I could go and say YouTube okay, please give me some results from internal AB tests that you have run about these and these things for example when introducing this feature. I said was it would establish some form of accountability regarding experimentation internally. So I think here. The idea being interventions my hundreds that that's limited and the question is, how about asking questions regarding internally be tests and internal experimentation by platforms as a researcher. I think that's a really good idea. My gut feeling would be based on the legal test. This kind of data should fall under the data access request provision, provided that you know that the AB test has taken place. It's always a little bit the difficulty and also one of our criticisms of the data access provision is that you kind of already need to know what you're looking for when you're making the request, because unfortunately there is this relatively strict purpose limitation that your research has to relate to those systemic risks, but it doesn't say that you can only request for example content moderation data. I would say the results of AB test would fall under those provisions. It's going to be a difficult situation, perhaps if you're saying, I assume that AB tests have been done on this particular question like for example how does this feature affect miners on TikTok, for example, or what if the platform says we haven't run those tests and it might be a case of an overly specific denial. So I think in that case it's going to be very important to have good relationships between civil society academia and the regulators because the regulators do have some pretty far reaching powers in terms of auditing the companies, but yeah I think in principle that kind of data would fall under the shared access provision. Martha, do you have any insights on that? I think really that as more as the question relates to the issue of systemic risk as India, it will be. So if you have AB testing on, for example, issues of like recommendation prior to election campaigns, they would say it's easier maybe as you are asking for AB tests maybe for certain types of new advertisement mechanisms. I think also especially arguing with the digital services coordinators to really show the impact of this data on kind of research for the on systemic risk will help there. So perhaps to add to that I mean traditionally we have dealt with this problem when it comes to governments and public institutions so we've done quite a lot of litigation around access to information and we also use access to information laws very actively in our own work. And there you have the same problem you need to know what to ask for. And in that respect, we found that open sources intelligence is a very important tool. So for example you could use this public database, the same way that journalists and researchers have been using Lumen for many years to try to figure out if there is anything that looks fishy. And then you can try to build research questions based on the information that is already out there. So for example if you're researching advertising there is going to be a fairly strict transparency requirement on advertising for very large online platforms so you could use that database to build some hypotheses and then ask more targeted questions as part of this research data access provision. The one thing that I would add to this great question is that as Felix points out, Lumen has to an extent been used to this and one of the things that we've observed that's critical for successful research is at least some level of data monitoring. Now I have every hope that the DSA is going to require some kind of standardized formatting, but it's easy to imagine the platforms making a plausible case that will our content moderation decisions are based on XYZ and therefore we have to give it to you like this, whereas the other one says no it's 123 we have to give it to you like this and there won't be any way to do any kind of aggregate analysis across. To the extent you're able to we're surely doing our best. There has to be some standardization because only then is the kind of research that you just propose really possible which is. I've noticed a pattern, whether it's through individual research or some sort of, you know, machine analysis both of which have been done on Lumen, and then I start to ask the questions. You know it's almost like a public records request. Oh, I found five or six that look the same. Please give me everything pertaining to that material or these requesters, which our experiences is often where the really good stuff is who sent the request to begin with and if you can establish patterns there whether name or, you know, legal address that sort of thing that that's where you will start to see patterns but I my personal read and I feel like I reread this document once a week. I don't see anything yet that would allow you to say specifically show me your internal testing. Yes, a user had already made a request for an explanation of a content moderation decision that coincidentally was part of that internal testing and then you could say something or another show me all analogous content moderation decisions. Well, I would say you have to differentiate between kind of the public database which is limited to content moderation decisions and the research data access request which I read as being broader. Then again, yeah there's the question, how do you know what to ask for and then to what extent can you ask for a specific format. And that will be I think very much up to the digital services coordinators so I mean we've had some of those voluntary themes in the past like social science one which I think ended up not being that useful also because of some of the restrictions around the format like if you have to physically go to the offices of one of those companies and then you can look at certain data but you cannot combine it with other outside data sets or you cannot take the data home with you or work across different jurisdictions with it that might very much limit the usefulness of the provision. So I think it will be very important between now and when this becomes applicable next year for researchers to also engage in this process so the European Commission is planning to do a public consultation. So on this research data access like how is it supposed to work in practice, and that is going to happen sometime in the next months or early next year at the latest, and there I think it will be very useful to bring in these practical experiences for example how the AB testing should be covered in it because it might be one of those open questions, or also how you need access to the data like do you need API access, or is it useful to get a hard drive set in the mail that you can then do what you want with and how also to combine data protection and trade secrets protection with the research interests in a meaningful way. So I think yeah there are a lot of open questions around that but there are also groups that are that are forming to try to inform that process and give advice to the European Commission. I don't want any other questions but I'll come into your attention paragraph 96 and the recital which I think gives sort of the framework within which such a request for AB data might take place. Please. I was wondering if you could respond to the requirement that only academic researchers or people affiliated with academic centers are allowed access, because the platform and accountability and transparency act in US has taken a slightly different route where they've allowed NGO researchers and there's upsides and downsides to both approaches and I was wondering if you could respond to that. Yeah, so we actually fought for journalists and NGOs to be included in the data access provision. And I would say for NGOs we were relatively successful because if you look at the definition of a research institution it doesn't have to be a university. So it can be an NGO, but it has to be basically doing public interest research so that means if you have an NGO that is kind of working on one of those systemic risks, and the research is required for the NGO to fulfill its public interest mission then they should be able to make a research request, but a company would probably not be able to do so. So that means for journalists which are usually, you know, part of for profit newsrooms. This is probably going to be quite, quite limited so it might be more, you know, which is a shame because we know that a lot of the important stories that have come out of Lumen have been published like in the Wall Street Journal and journalists have been very active in using that. So they might have more difficulty using this but for purely kind of public interest NGOs that are not affiliated with the company, they should be able to make use of this provision and it's definitely something that we also want to support. So if there is an NGO that has the in-house data analytics capacities to make use of those data, we would also be very interested in testing that in court because they really should be covered. And where journalists are concerned that our part of the company, they can still make use of the database of content moderation decisions. And I think it's worth noting that it also goes further than the Lumen database because it's not limited to content moderation decisions that are triggered by the notice. So if a platform makes content moderation decisions out of its own initiative, for example, to enforce its terms of service, they do have to provide an explanation to the user. So like if they block your phone uploading something, if they block your user account, or to some extent, even if they engage in shadow banning and there again you have the question, how do you know that this has happened, but at least according to the law they will have to provide an explanation for this. And we'll have to publish this information in the database which would be accessible also to journalists. Other questions. You can take one more from the in person audience and then need to go to the sky lounge in front of her, please. Great. Thanks. I have an affiliate here. Question I have is about. Maybe it's a question for women and then ultimately could be reflected elsewhere. So the content moderation decisions are adds a decision of, you know, based on policy internally. And I assume that sometimes what happens is you have research that says, Hey, this policy is clearly their policies are insufficient because, you know, all of we're seeing all of these things happen and they're bad for some of the policy changes. And then you'll start seeing something else after that point. Is there any way in the Lumen database to see when a policy has changed, can a corporation or company decide to tell you that, so that they can show that hey we've actually changed something that's updated it's different from before which would essentially help them right if they're under fire, or is that part of the kind of game of the investigation and the research which is identify these things but not ever really know what's happening and what the constant moderation as a result of. So that's a great question on several levels I should begin with the disclosure that Lumen because of its history is a database of external requests to remove content we don't have anything that would be characterized as an internal content moderation decision we could ask that a lot. We'd love to, and that's one of the reasons that we're most excited about the DSA is because it's going to include all of a sudden, this gigantic, you know, dark data set of content moderation decisions so to the extent that that's your question I don't have any explicit insight but with respect to the broader, what if policies change. And this gets to what Felix and Malta described as you have to know what you're looking for before you look, because it's a it's there in the data, but it's usually not made explicit. So, for example, specifically recently, Google was under severe legal pressure in Spain. For what the Spanish digital protection authority said was illegally sharing personal identifying information by Spanish citizens and implicitly of all you citizens across the Atlantic to Lumen as part as part of their sharing with Lumen. I felt surprised to hear that because nothing had changed for this way and so without getting too deep into the weeds what it turned out they were talking about was EU citizens attempting to exercise their rights under the GDPR. And I should say that, unfortunately, unfortunately, depending on your viewpoint, Lumen does not have any GDPR based requests. Regulators in the EU quite sensibly feel that that's kind of a snake eating its own tail thing that the request to have something forgotten is then publicized somewhere else that sort of defeats the purpose. We think that there's a way to study those and we're advocating for that, you know, in a safe way but we don't have those. Especially in the earlier days of that regulation. People would say I want to send the GDPR request and they would go to Google, and they would find a form, and they would fill it out thinking that they were sending a GDPR request but in fact they would be sending a copyright request at least according to the form, which would be shared with Lumen. And so eventually this percolated out the Spanish DPA said there's a whole bunch of these notices in Lumen which we believe inappropriately contain personally identifying information which really was usually just someone's name and a URL and so they find Google 10 million euros. A large but possibly not large to Google, some of money. And so what Google there was they came to us and said okay we just we've just been found to have done this and we need to change the way in which we send you data. So going forward, we're just not going to send you this category of data which we have previously sent. That's a policy change. It's one you would immediately notice if you were someone already in the database saying every week I want to see all the notices from Spain, because then they would just go to zero. But it's not something that would be explicitly announced as a change because it's sort of. It's definitely not a lie but it's it's an action of a mission rather than deliberate activity and so I think that characterizes a lot of our research that if you're already paying attention you'll notice shifts, but there's no public statement of a policy shift, whereas with the change of commission, we're usually quite public about it we're thrilled to announce that Google is now going to begin sharing takedown notices having to do with counterfeit goods. We've created a new notice category you can find it here. So I think it's going to be pretty easy to notice but that subtractions or even worse, labeling shifts, like start calling this local law instead of other or something, something, something trivial like that will be much harder to notice unless you're already paying attention and so I hope that people will set up I guess. What would you call it like background monitoring like we're not actually like going to hold it but we just sort of we're watching. Yeah, we're looking for the changed changed. I think these kind of policies will play a huge role because all of these very large platforms will have to design their own risk mitigation measures so they will all be forced to do something on things like this information on elections for example if that's something that happens in the context of their platform, but the exact measures will be largely left up to them. And so the hope of the EU legislature is that researchers will use these data access provisions to monitor whether the risk mitigation measures are working. So there's also like to to an extent that the EU is putting a lot of faith in all of you to help them do their job because like the public regulator does not necessarily have all the data analytics and research that are needed to really see whether this risk mitigation measures are working so that's, I think the policies will play a huge role. That's well put here we go to the Skylabs to you. Left on the clock, hopefully we'll read this question correctly, but going back to potential research questions this individual is asking about what your thoughts are on asking answer on state run internet referral units and formerly asking for takedowns. Specifically systemic risks of pertaining to freedom of expression discrimination. Since most of those requests appear to focus on minority groups and could consist of indirect state censorship. They also want to add that this question was inspired specifically about Facebook quote unquote voluntarily taking down drill music videos, following the London Met police and formal requests. I mean, I think you could ask questions around that because it relates to the freedom of expression of users. And I think these kinds of requests should be even easier to identify because there's probably an open line of communication between state authorities and the platforms already so I think that would be a good example of what you. My follow up to that would be, I think that that's an extremely important potential risk to be of which to be aware, but that given the provisions of the DSA I think that it's going to mitigate that kind of risk because those requests will in theory be in the database that there was a request to be assured of a request for a platform to exercise content moderation according to its terms of service. The piece that I don't know is that, as I understand the database, personally identifying information of requesters is meant to be shielded, and that makes perfect sense. Historically, at least some of the platforms have drawn a pretty bright line between agreeing to shield, if Adam Holland or Felix Rita sends a request versus government of X or ministry of X center they actually want to be quite public about that so I could see in fact this being a phenomenally transparent way that would actually lead to far fewer uses of that kind of soft power. I mean you're the GDPR expert but if it's like a public authority saying please take this down that personal data. Yes, I think so, especially when it's really on in individuals who you don't know. So it isn't as easy I would say under GDPR but at least you could ask about like extra information on how many cases or are there some specific issues and so on and you can also always ask if you are, if you are connected on your own tools, then you can also ask because then you're kind of opening up and GDPR isn't any longer because it's on your own information kind of so but it could be an issue maybe where you also have to go to court to see if there's a line of GDPR maybe protecting such issues. Yeah but even if certain data is redacted from the public database you might be able to ask follow up requests through the data access provision and provide a data protection strategy to the regulator and say this is how we're planning to protect this data if we get access to more. So these are some of the questions that we would really like to test out. And also really help with the research us to draft such concepts to. So definitely if you want to make use on this you don't necessarily have to be a GDPR expert we're also kind of looking to bring people together with different types of expertise like so we have the kind of legal expertise we lack the data analysis expertise. Part of what is happening between now and when this all comes into effect is also to bring together the people who might be making good use of those provisions and yeah make access requests to the platforms that would actually hold up in court would be accepted by the regulators and would be useful to the researchers while also of course protecting users personally. Well that wraps up for our virtual audience is anyone that we didn't get to we will forward these questions to our panelists but now we'll hand off the panelists to close out the show and again thank you all so much for joining us today. Thank you. What would you like to make your one final pitch for contact and participation. If you're already here the best thing is just to stick around I mean we have this room for a while longer so if you're thinking of a research issue that might fit with this please stick around and we can have more targeted discussion on that. Thank you so much both for being here this is exactly the kind of thing that I believe Bergman Klein exists to do the kind of collaboration we want to have and this is just. If you if you haven't got it yet. This is just an absolutely mind blowing zeitgeist changing phenomenal new piece of legislation and no one's really sure what's going to happen yet. Thank you.