 What's app could soon be banned in the UK? What concerns have been raised over the company's privacy rules? Back with its controversial privacy policy, only this time around, the Facebook-owned app is allowing readers to vote through the policy changes. It is often said that chat control is a threat to our privacy at its mix and to end encryption illegal and allows the government to check all messages shared within the Europe. This is false or at least misleading. The chat control law is indeed a serious privacy threat, but to understand why we have to dive in into the complexity of the law. In this video, I will first give some context about the law and its timeline, then I will explain why it's actually really easy to think that it's not a big issue, and then I will explain why yes, it is a terrible law and finally I'll explain how we're going to save our privacy. So let's start. So context, the story begins in 2020. Europe notices a significant increase in child sexual abuse material evidence. In fact, just between 2020 and 2021, one year later, online sexual exploitation cases raised by a factor of 10 and child sexual abuse internet videos by 40%. They immediately go ahead and approved a so-called temporary legislation to allow chat message and email service providers to check messages for illegal depictions of minors and attempted initiation of contacts with minors. Immediately services like Facebook, Messenger and Gmail start looking for evidence of child sexual abuse material. This is an opt-in legislation. That is, it only gave permission to those companies to search through the messages for that kind of imagery, but it did not make it required by law. One year later, 2021, the European Parliament approves a legislation called the high privacy derogation, no longer temporary, that again allows message and email providers to search for child sexual abuse content. It's now necessary to understand how Facebook and other companies are checking for this kind of content. A great example is Microsoft. Obviously, having a large public database of child sexual imagery to check for matches would be insane. So Microsoft designed a tool called PhotoDNA that relies on hash functions. An hash function takes an image and gives you a number. It should be very easy to find this number given the image, but it should be instead be impossible given the number to go back to the image. This way, every time police identifies a sexually explicit child video or photo, only the hash of that content will be in the database and when you send a message over Discord, as an example, the hash of your message is checked with the hash of the illegal content. And yes, Discord does this for every single picture you send. It's not a random example. Hash functions are usually not unique. Let's say if you have different inputs, like two different pictures, it might happen that they have the same output or the same hash. This is part of why it's so hard to take an hash and go back to the original image since there's no one original image but rather many different possible ones. However, it is extremely unlikely that two different images would have the same hash in practice. So we are safe from false positives to a very high extent when comparing hashes. Now, usually it is a requirement for a hash function that changing the input slightly will result in wildly different output. This makes sure that it's really difficult for two similar things to accidentally have the same hash. However, such an hash function would be wildly useless in our scenario, as it would mean that changing the image slightly would result in a completely different hash. So you would have to just go ahead and change one pixel to be safe from these automated controls. Because of that, further DNA and similar tools use a different kind of hashing function, which is called perceptual hash, which actually gives us similar hashes to similar images. This will be particularly important later on. 10 years later, in 2022, the European Commission decides that they would like to have this kind of control on all different platforms. This proposal is what we now call the chat control legislation, and it actually introduces much more than just making the current checks mandatory. The Council has had six months of workshops to define an initial draft, which might be different compared to the one proposed initially by the Commission. A group of the European Parliament, called LIBE, which is a Committee on Civil Liberties, Justice and Home Affairs, also started working on its own draft. After this stage, we will have the trilog discussion that will bring the three different drafts back into one. The cool thing is that we're very close to a Parliament draft, and apparently we have an agreement already, and the vote will be just in a few days, on the 13th of November. That's one thing though, this video is not sponsored. Nobody pays for it, and yet it took one week to research and write, and now I'm recording it with some quite expensive equipment, and the editor is gonna spend days and days editing it. Ad revenue alone isn't nearly enough to cover all the expenses, I'll probably make around 30 bucks 50 if I'm lucky. The only reason this video exists is your donations. Above my head, you should now see a progress bar with the money I need monthly to give this whole thing running, and here's how much I currently have. I really want to thank everybody who donated, and if you'd like to, I've got people, LibrePay, Kofai Patreon and so on. Anything is appreciated. So without further ado, let's start talking about chat control. First of all, all chat messages and email services that do not get any remuneration, even from advertisement, are affected by this legislation. This means that open source projects, such as Katie's Matrix client, Neochat, are certainly safe and won't be changed at all. Note, however, that this legislation does apply to services that are not based in Europe, but still operate here, such as Telegram, WhatsApp and so on. This legislation includes telephony, email, messenger, chats, this includes video games with in-game chats and video conferencing software. Just in case you don't know, almost all chat software require a middleman that takes your message and delivers them to your target person. So I give Telegram the message and Telegram gives my girlfriend the message. Lots of chat services encrypt the messages in these two steps to make sure that nobody can read them. So I encrypt and then send to Telegram. Telegram decrypts the message and then encrypts it again differently and gives it to my girlfriend and girlfriend decrypts. And that's it. This means that Telegram can actually read my messages and my images. And the same goes for platforms such as Discord. This is how they're able to check your picture with PhotoDNA. They are able to see them. However, some chat services such as WhatsApp actually have a different type of encryption called end-to-end encryption, where only the recipient of the message can decrypt it. So I encrypt my message, give it to WhatsApp, WhatsApp cannot decrypt it and they can only give it to my girlfriend as it is. And only she can actually read it. This allows for a much higher standard of privacy and is highly recommended to say the least. So far, all systems using PhotoDNA were not end-to-end encrypted. However, the draft legislation says that we should check for end-to-end encrypted messages as well using similar techniques. However, fear not because this is actually possible. Basically, instead of having WhatsApp, check the messages because they can't. This hashing would be done directly on my device when I send the message. There are a couple of ways to do that. First one, after I click send, my phone hashes the message and sends the hash to WhatsApp. This is safe because they cannot know what your message is based on your hash. They can only know the hash. And now that they do have it, they can just check it through the database. That way, when I install WhatsApp, the application automatically downloads my phone, the database of all hashes. After I hit send on my phone, my phone hashes the message and directly checks within my device through the database without even sending any data to WhatsApp at all. On top of all of this, the commission wants to check companies for child explicit content even through hosting websites, which includes web hosting, social media, video streaming services and cloud services. In fact, Google Photo already scans your pictures for child pornography. This shouldn't come as a surprise, but this legislation would make these mandatory for companies like Nextcloud as well, or rather companies using a Nextcloud install, I think. However, the commission also decided to bring all of this to the next step. This method I've talked about, PhotoDNA, only allows search through images for known child sexually explicit content. Instead, the commission would like for it to be mandatory to search for unknown content as well. From their point of view, this move is pretty obvious, I would say. And in practice, it would involve a machine learning-based application to automatically flag content as problematic. Again, in the context of end-trend encryption, all of this would have to be done directly on your phone to make sure that you're not sending any data around. Not only images, but the legislation would also like to check for grooming, meaning that all text messages should also go through a text-based machine learning model that would automatically flag grooming. How reliable are these automated tools, though? Even Microsoft has one, and they report accuracy of 88%, meaning that out of 100 conversations, only 12 are false positives. Note that these are manually filtered out by Microsoft and will not contact police enforcement purely based on these automated tools. Finally, there is a new requirement on app stores such as Google Play or Apple, whatever, to verify the ages of users and block children age 16 and under from applications that could be misused for grooming purposes. I will note here that I was unable to understand, at the time of writing, the exact meaning of this age verification. Some sources expect a very hard and formal age requirement, which would require all Google Play users to identify themselves. However, other sources think this could refer to some lighter kind of verification, though I'm not sure what that would be. I will get back to this in a later part of the video. Nonetheless, this is a very brief explanation of all the things that are contained in the chat control proposal by the European Commission. As you can see, it seems like the end-to-end encryption is granted, guaranteed. All data processing is done directly on your device and privacy is safe. However, the situation is not as good it seems, as we're about to discover. And really, this should come as a reminder that understanding policies is always extremely complex. Let's immediately start speaking about end-to-end encryption and this idea of looking for hashes directly on your device or even sending those hashes to a third party. This concept is called client-side scanning, CSS, and has some major drawbacks. If the hash lookup is done on your device, there's obviously technical challenges. Your phone would have to store the hashes of all known child sexual abuse media. This won't be an issue from a legal point of view because, again, it is impossible to restore the original content after you hashed it, but it does require your phone to have a big database inside of it, which has to be constantly kept up-to-date. On top of that, if the hashing is done only on your device, you have to make it very hard for users to disable that component by, I don't know, deleting some files or killing some processes. Not that, but you get the idea. However, sending your hash to anybody has a much bigger privacy drawback. Let's say that I write a message that says, I hate the Italian government. My phone will turn that into a hash and send it to WhatsApp. They won't be able, in any way, to restore my initial message from the hash, unless they guessed it. So what they can do is say, Hey, I think we should check for everybody saying, I hate the Italian government. Then they can take the hash of that wording and check for the resulting hash on all of their messages. My message would have the same hash and they would be able to know that I sent that exact message. Now, it is insane to check all possible combinations of text messages with this technique, meaning that, yes, the vast majority of messages will never be understandable by WhatsApp, but they can still track for certain messages that they would like to know about. And really, this is working as intended. This whole hashing thing is supposed to make it easy to WhatsApp or whatever, to search for all messages for a certain image or text that they decide. The chat control legislation says that this should only be used to search for child sexual abuse material. But obviously, this provides no guarantee that our privacy is safe, because it is technically feasible for them to search any kind of text or image, not just that. If you follow politics, it's easy to see how we should never allow the existence of tools that can technologically be used as citizen tracking devices, because the legislation said tracking users is illegal. This is because the very first authoritarian government might want to extend that from, I don't know, from child abuse to revenge porn. And that's a good cause, right? And then from revenge porn to illegal material in general. And then from illegal material to anti-governmental material and so on. It's really not that hard to find a government that would be more than happy to use these kind of tools to search for opposition messages. Just in Europe, we have governments that are slowly degrading their democracies, such as Belarus. But even the Italian government has made it clear that they would not tolerate any kind of public anti-governmental ideas. What if the hashing is done on your device then? Aside from the technological issues that I've talked about, there are a couple of big challenges. The first one is what happens if your device finds that you're trying to send child sexual abuse material? What the child control legislation would like to happen is that the nasty messages would have to be shared publicly with a relevant authority that would review them and potentially open up an investigation on you. This has a small issue. Hashing doesn't hash hushingly and you accidentally get flagged for child abuse. Happens. Keep in mind that we're talking about a billion of messages every day. In those scenarios, some of your messages sometimes might be accidentally shared with third parties that would be able to review them. This is the privacy concern. There's also another much bigger challenge. As I've said before, you cannot retrieve the content from its hash. This means that you cannot retrieve child sexual abuse videos from the database that you're given. Thanks God. But it also means that we have no clue whatsoever about what's actually in the database. If the entity that puts together the database decides to add some images that are not child sexual abuse content, such as revenge porn images, we would literally have no way of knowing that. The entity that works on this database would have obviously to be opaque because the alternative is they would have to publish every piece of child sexual abuse video. They can't do that. So we would simply have to trust them blindly that they are not searching for anything except child pornography. Even though they can and we wouldn't know. Even if the entity is extremely trusted and has some mechanism in place to make sure this won't ever happen, this database thing adds up a pretty big attack surface for hackers. As I said, the database would have to be constantly kept up to date. If somebody wanted to read my end-to-end encrypted messages, they could try to modify this database, either pretending to be this above entity and sending a fake update or they could directly modify the database on my device if they get access to it. And if that happens, if the database is compromised in any way, then my device would start sending some of my private messages to the above mentioned external authority instead of sending them end-to-end encrypted to the person I'm talking to. Sure, we can try as hard as possible to make it sure that none of this happens and maybe there is some integrity check for the database and such, but it's just all about the surface area that bad actors can use to attack us. All this assumes that the hashing function works as described. The funny thing is though, it doesn't. It's pretty easy to write a normal hash function as I've described it, but remember that we are not talking about normal hash, we are talking about perceptual hash, which requires small changes in input to be small changes in output so that we can check for similar images and not just perfect matches. The cool thing is that an open letter by a hundred of university teachers clearly state that even after 20 years of research, no such function exists. Quite the opposite, we have shown that all perceptual hashing functions we know of today won't work. This is because it is virtually always possible to make a small change in an image that will result in a big change in hash, meaning that bad actors can always slightly change the image in a certain way and go unnoticed. On top of that, it is always possible to create an image that looks completely normal and yet has the same hash of a child pornography image, meaning we can actually generate images that we are certain will result in false positives. It's pretty easy to see the issue here. A bad actor might generate such an image and use it to frame an innocent or float the low enforcement agencies with false positives. Both of these attacks have been achieved successfully against both photo DNA and Apple neural hash. With what's Apple neural hash, you might ask? Well, basically, Apple tried to implement client side scanning CSS and they actually did introduce it saying that it was the very best and the very latest technology as always. And they actually removed it after a couple of months. So yeah, and so far that was the only time only time CSS was ever attempted because photo DNA is not used with end-to-end encrypted messages. So yes, Europe wants to make mandatory to use a technology that has only been attempted once by Apple and the field. The only way to try to avoid all of these issues is to design an hash function and keep it secret. Just don't tell anybody how it works and nobody can do these attacks, which is not going to work. And I really hope that you can see why without me explaining it. The last important quote from the open letter regarding this is as scientists, we do not expect that it will be feasible in the next 10 to 20 years to develop a solution that can run on users devices without leaking illegal information and that can detect known content in a reliable way. That's quite a statement. All of this is about the search for known child sexual abuse material. But of course, child control also wants to introduce search of unknown content using machine learning tools. Now, AI tools can be trained to identify certain patterns with high level of precision. However, they do routinely make mistakes, which makes them a terrible idea once we realize what scale we are talking about. Even scanning the messages exchanged in Europe for one service provider would mean generating millions of false positives every day. And even if this machine learning model runs locally on your device, each false positive means, again, that your message won't be private at all and will be instead be shared with external and they might even decide to contact police enforcement because of that false positive. Again, an amazing quote, this cannot be improved through innovation. False positives are a statistical certainty when it comes to AI, especially when we're talking about applying it to all messages sent in European Union. Because all of this, even though end-to-end encryption is technically preserved, even if chat control, the CSS represents a privacy risk big enough to put at stake the entire concept of secure communication because you can never know whether a message of yours will be actually sent to the recipient or whether it will become a false positive and be instead sent to somebody else who might contact police enforcement because of it. We cannot allow that to happen. Honestly, it feels like I could just end the video here. This is already pretty darn bad, but yeah, there's a couple more things to talk about. Very quickly, I will point out that another major issue of all of this is money. It is, to me, very unclear who is going to perform the manual check of the messages that were detected through these hashes or machine learning tools. This might be done directly within the companies providing the services who would have to hire people whose job is to look at child sexual abuse images all day. And we have articles on articles on how difficult and unhealthy those job environments are, especially if there's a lack of proper compensation and mental health services. But it might also be done directly on the low enforcement side as they already have groups designed for it. Regardless, it is pretty clear that this will require an increase in public spending to make sure that there's enough stuff to go through all the potentially false positives that were raised out of the billion of messages we send every day. Right now, 80% of the police reports that reach police are criminally irrelevant. And I expect that number to skyrocket if this legislation passes as it is now. Secondly, we already have reported cases of people wrongly accused of having child pornography. As an example, there are at least two cases of parents taking intimate pictures of their son or daughter to send those to their doctor during COVID lockdown. They lost all access to their Google accounts, email, calendar, even internet connection. And their local police enforcement actually investigated them for months, which after which they contacted them to say, hey, we found nothing illegal. You just took a picture for the doctor. And yet after that, Google refused to reinstate their accounts and automatically wiped out all of their data after six months. And that is the best case scenario. We do know of people that were wrongly accused of using child pornography and who took their lives. Thirdly, if the age verification rule ends up to be intended in a somehow strict manner, it will mean that it will be impossible to use any kind of online messaging platform anonymously. Given that app stores have to check the age of their users, they will require some sort of document or ID before being able to access any message platform. And we are particularly worried for compulsory identification, the collection of biometric data and the interference with encrypted communication. What could be preferred as advocated for by Durkin, no, no, Durkin Dershoot Sudbund, okay, I tried, I tried, is that large advertising financed platforms automatically flag children as such, given that we've seen they able to identify children automatically and young people as users for a long time. We might also want parents to create a smartphone accounts that are specifically flagged as children accounts through the voluntary age declaration. This way parents would be certain that their children won't be able to download any kind of dangerous application without their parents consent. Please note that I've only scratched the surface of the criticism towards control and I will leave quite a lot of resources in the video description that will tell you more about this proposed legislation. However, we are already pretty deep in the video, it's 30 minutes now and I'd like to start talking about how likely many of the issues with chat control are being solved through the European legislative process. I talked early on about how there's actually three drafts of the chat control. One is the original from the commission and then there's a draft from the parliament and one from the European Council. Then all three are merged together. Luckily, the drafts of the parliament and council appear to be significantly better compared to the commission one. Let's dive into that. The council decided to keep roughly everything except the machine learning part, so all of that gone. The parliament, it's a bit more complex. There seems to be a deal in how the drafts will look like, which is great. And again, the vote about it should happen in a few days, like on the 13th. It is worth noting that part of the LIBE group is Patrick Breyer, which is not pronounced like that, probably, who has done an outstanding work of communicating the issues of this proposal to the public. He's part of the Greens-EFA group and has been elected through the German Pirate Party in 2019. This is extremely impressive, considering how the Pirate Party is a rather small party who still manages to be presented at multiple elections in multiple European countries. And the best result that they ever achieved is 7.7% in Luxembourg. They managed to elect four people to the European Parliament in the last latest round of European elections. And I see all of this, not because they beat me or anything. In fact, I am part of another political party, so I can only lose by talking about them, but rather, because if you care about all of this and want some privacy-minded people to be elected and be able to negotiate better legislation, we do have an European election coming up next summer between the 6th and 9th of June. My birthday is in there, by the way. If you live in Europe, it might be worth it to check whether the Pirate Party will be in your elections and check whether you think they're offering a good candidate. But back to chat window. The draft that hopefully will be voted immediately states that all end-to-end messages services don't have to follow the legislation, so won't be covered by this. This means that CSS, with all of the issues we know it has, wouldn't have to be employed at all. It also asks for telephony and text messages to be removed from the bill, even though messenger services are still in there. And it removes the machine learning part, but only for grooming. There are still proposing machine learning to be used to detect child sexually explicit content in images or videos. The Parliament drafts also removes edges verification entirely, and this immediately makes the bill a much more sensible from a privacy point of view, especially given that end-to-end encryption is kept out of it. However, even if this draft is accepted, it doesn't mean that this will be reflected in the final draft, especially given that in my humble opinion, it would become hardly of any use. Even extremely common chat services like WhatsApp are end-to-end encrypted, and it's really easy to have them in telegram as well. So you can also download end-to-end encrypted chat services like that. I guess that people who use and share child pornography are just gonna very easily switch to any of those messages or most likely just keep using the one they're using, because probably it is end-to-end encrypted already. I'm not saying this because I agree with CSS, but rather I'm saying that, given how important end-to-end encrypted messages are to the intent of the legislation, I wouldn't be surprised to see them, to see the Parliament draft to be ignored in this aspect. We'll see. I think this is the longest scripted video I've ever done. So again, links in the video description, both for sources and donations to the channel, which would be extremely, extremely appreciated. Thanks here for following all 40 minutes of recording, and... See ya. I do videos like this quite often, lately.