 All right, there we go. So I'm Toby. I'm from the lovely city of Hamburg in Germany And we have this project called privacy score.org and for the next 20-ish minutes I want to introduce you to the platform I want to show you what it does and what our aims are and I hope that you know After this session, you will know what it is and you will like hopefully like it Maybe you will even send patches or buck reports that be great. So let's start Privacy score.org is a service for investigating security and privacy properties of web pages. So what does that mean? Well, there's several definitions, right? I mean, depending on who you ask, you get Different answers as to what it means for a website to be privacy friendly or not. In our case We were noticing that the city of Hamburg's webpage You know, this is the website of the city of Hamburg and this is the website for social welfare So if I'm interested in social welfare, then I visit this website of Hamburg DE You can't notice but that's like the city where I live in and on this website You have multiple trackers like plenty These are the companies that also know that I'm interested in social welfare And I mean, here in Europe it's probably natural that this is a more tangible topic Like it's a more privacy, you know, this topic needs more privacy than other topics I was recently in Cuba and I think the concept of social welfare is a good thing And people are not necessarily concerned about people learning that you're interested in social welfare because everybody does So in Europe, however, it's a bit of a touchy subject and you might not necessarily want other people to learn That you are, you know, not among the top 1% or whatever and that you are interested in social healthcare So these companies also learn that you are potentially poor or whatever And the question we had is, is that the new normal? Like do people do that? Is that like the world we're living in now? And we didn't really have a tool to answer this question Because I mean, you needed to investigate all these websites of, you know, the public sector and check whether they use external tracking companies or not So we built this service for scanning the websites and then you could argue, well, there is so many services already that scan websites, right? There is like the most famous one is probably the SSL Labs one which scans your TLS configuration of your web service And there is like a privacy, more privacy focused thing from the data protection agency of non-confuse I think it's Sweden, but I'm not entirely sure These services exist and they are, you know, they're providing their service and it's great And there's a few others, there's one from the Fraunhofer and there's yet another one which sort of tries to make propaganda for using TLS on your web service And these are all great services and they're all good There's more advanced ones, but they target the operators of the websites rather than the users Or the services use a more predefined scheme of getting points to websites, you know, of getting or of ranking the results This is the, from the Mozilla Observatory, like the scheme of how they give you points or deduct points And our idea was to have public benchmarks which make it possible for website operators to get a benefit out of making the website more privacy friendly So how would that work? The idea was to have public lists, you know, you would group similar websites Imagine all German cities in like one list and then you would see these websites being ranked And hopefully, you know, a lower ranked website would see that they could do better, they could, you know, go to the top of the list When they would use one less or one tracker less or something So that you have a publicly available list of your, say, performance, how you compare to other websites And then by implementing some more measures, you would, you know, gain more points and thus be better in the ranking And of course, these cities in Germany is just one example You could easily think of other examples like, you know, health care providers Or because we're at farce them of new Linux distributions or desktop systems I'm a GNOME guy, by the way, so I have a slight bias towards GNOME And I try to sneak in some propaganda, you know, and fall in favor of GNOME every so often So you could do that And this was the idea to have these lists of websites and to have people compare, you know, these websites among themselves You know, what we do not want to do is do any kind of pentesting or aggressive sort of security checks Like things that would alter the state of the server, you know, are not within our scope And also, it's legally more problematic in everything We intend to have, to make the user change the ranking, change the results You know, because for some people it might not necessarily be interesting to, I don't know Know whether these servers are, like, located in Europe This is especially true for American companies, you know If you're comparing American websites then whether these servers are located in Europe or not Is not necessarily interesting for the ranking of the websites And our goal is to let the user configure their ranking and to enable the user to make their own informed decisions Based on the data that we have collected We're not there quite yet, but I mean, hopefully we'll get many new contributors soon And then we'll get the patches to make that happen And it's a free software, of course, because we're at Fasten, right? This is a GPLv3 software, you can download that from GitHub and install it and run it And, you know, improve and share and hopefully make the world a better place By scanning the websites and telling them how they could improve, you know, the privacy on the web And, yeah, so please check it out, PrivacyScore.org is the main website And you'll find it easily on GitHub And with this tool to coming back to the idea We now have something that can scan various lists of websites and compare them In order to, you know, answer the initial question, the research question Of whether HamburgDE was more special than other cities And there's actually this list, it's on PrivacyScore.org If you click through the website, then you can see that we did add the city's use tracking services Well, we have these few lists We do have new sites because it's more or less common knowledge That publishers, like news publishers, do use more tracking services than other, you know, websites Categories of websites We also have the new Linux distributions from distrowatch.com And you see we have attributes And the idea is that you can sort of correlate the attributes with their performance Maybe bigger companies have better privacy properties than smaller companies Maybe distributions, older distributions are better in terms of privacy than newer distributions Things like that And so you can create these lists, everybody can create such a list And hopefully these lists are annotated with a lot of properties And we have projects with a standard FOSDEM So you could, you know, click the open the list and see the results And check how many of the projects that we have here at FOSDEM use tracking services Turns out all do but two So all the, I forgot the exact number, like 100 or so projects being present here All of them do tracking but two The FSF, yay, and Viking hosting And you can, of course, scroll to the very bottom of that list And, you know, see where this project is here at FOSDEM Where they have their stand and then approach them and tell them Well, listen dude, if you would enable, I don't know, TLS 1.2 or so Then you would gain two points, maybe you should do that And then your ranking would raise, like you would be better in this list So please check it out, it's like live right now You can check it out in this very moment Then hopefully if you know people involved in these projects And, well, and you know who can make their websites better, more private More privacy friendly, more privacy friendly Then please go there and approach them and help us making the web a better place And we have, because I mentioned I'm a known guy We've also set up a list of desktop environments And you can compare, you know, which desktop performs how well in these tests And maybe you select your new desktop environment based on these results Maybe not What do we do? So I've now talked a lot about the idea of what we intend to do What we intend to achieve And now I want to talk a little bit about how we intend to do that Right now we have four categories of checks So we check four big things We have four pillars of checks that we do And one big check is the note tracking check That uses OpenWPM, it's a great software library, it's a framework It's essentially Firefox with a Selenium And it does various, well, checks itself It tells you how many requests the browser performed And how many of those requests were towards known tracking services And these type of things So OpenWPM we use here And then we sort of display the results in an aggregated and hopefully nicer manner Then we use test SSL for checking the TLS configuration on the web servers On the web server and on the mail server And then we check things like do they actually support TLS at all? Do they support TLS in recent versions? Do they, does the TLS stack have known vulnerabilities? And you would be surprised to learn how many servers out there actually Have known vulnerabilities that would be easy to check by simple update Of their open SSL library or something And these, we use these tools like test SSL in this case And display the results in a hopefully nice manner And then we check for basic attacks Relatively simple attacks that could be possible Or whether these attacks are possible at all This includes certain headers that should improve the security while browsing the web Think of the X frame options header Or think of the content type header So that your web browser defends certain attacks much Or that your web browser can defend against certain attacks more easily Than if the header was not set We also do some interesting information leakage checks I like those because, well, they tend to be more dramatic in certain cases This is some instances of people leaving their private key around We test for domain.com slash key. What is it? Or private.key or something, or domain.key And sometimes people just leave their stuff behind On the root directory of the web server without deleting it And then you may be able to catch that private information And of course, when you find the private key of the TLS certificate Then you can decrypt all the communication and that's no good So we do these types of checks as well And we are of course open to more checks And currently we're not doing all the checks possible Because we simply don't have the tooling to do that So if you can think of a certain check that you think is important Then we'll be happy to add it to the privacy.org platform So I've talked about that How does the actual ranking then look like? By now some of you have opened the website already We have the concept of lists So people are supposed to upload lists with the URLs And then optionally some attributes for each entry And then we show the results in these columns And the user can change their ranking based on their preferences By reorting these columns In the future we hope to have a more fine-grained sorting And filtering mechanism for the results Right now it's a bit primitive We're being academic so we're not necessarily the best software engineers We code up until the point where it works And then we probably need to go on to the next project So there's a lot of room for improvement That's the positive formulation of the situation So many patches can be contributed still Because it's not yet the pinnacle of software design and architecture So if you want to learn some Python and web and Django Then you can improve the project very easily There's some low-hanging fruit around Anyway, as I've said, we try to enable the regular user To make informed decisions based on the data So we're trying to display the information in a manner That allows users to make informed decisions We've opted for these red, yellow and green status icons And green with a tick mark is good And red with an axis not so good And the middle ground is the orange thing And we try to make an aggregate of the results available also So that you get a general idea of how well or how bad this list performs overall And this is the list And then you can get the more detailed results By clicking on one item of this list And then you see what checks have been performed And then you can hopefully learn how to make your results better With the next scan So that's the idea, at least, that you can improve your offering If you wanted to You can expand certain items to get a more detailed description of what the check is about And what the result means, how the result came together This is a PHP info We get this often Many people leave PHP info.php around And then you can say, well, so people learn that I run PHP What's the matter? Well, you write, that's probably public information Everybody does, PHP, everybody's sorry about it Nobody really wants to run it Everybody knows that, it's a terrible thing In this case, though, we see that it's a very old Linux version, right? It's been built two and a half years ago from now And this particular version is probably eight security fixes behind the most recent state So with this information you can infer Or the hypothesis is that with this version you have at least eight security vulnerabilities Because you've skipped eight releases, eight minor fixes So the attacker may have an easier game if they know that you're running this version Which is vulnerable against this or that attack And this happens very often And I think that website operators do not necessarily want to expose that They do that because they are installing this service or whatever And then once they're done they go on to the next project And maybe they've just forgotten to delete this file And maybe such a tool like privacyscore.org would be very helpful for them to, well, not forget about these things So we're trying to be helpful there and make the world a better place People might not necessarily be happy with us being such an intrusive Client and requesting all these weird or potentially malicious files And some people might rightfully ask, well, I actually allowed to do that With what gives you the right to open our website Well, this is sort of this sort of request that we get every now and then And because we're in Germany you cannot simply open a website You need to have this investigated whether you can And we did, turns out, yes, you can open websites But it's not as trivial as it sounds, right? I spare you the details But in this, like a couple of weeks ago, we got requests We got this, as I've said, we've got these emails How dare you to open our websites from companies who are ranked in lists And who are not necessarily happy with being out there in the public and all So it is German people ask the question whether it is legal to open websites And TLDR, yes it is Of course, you might not only ask the legal question, but also the ethical question Do we really want to expose the information that certain hosts are vulnerable or not To certain attacks, you know, because it also makes it easier for attackers to Well, compromise services, compromise the privacy of users And it's a valid question to ask And we've went or we've opted for saying yes, well, we hope or we think that the greater good is to Well, to make it easy for website operators to fix up their services And we do not directly show certain results, you know, which are very Which could be very or which could lead to an easy compromise of the target service And we try to, you know, be as helpful as possible without exposing information that would be used for attacking purposes only And of course, if you really don't want to be listed on the website then we respect that And we put you on a blacklist so that you'll never be scanned again And we expose that fact also, like everything, we try to be as open and as transparent as possible Because the actual main motivation is to make the web more transparent Because transparency is what we think is what we think is lacking in this whole tracking thing Because, as I've said, the initial motivation was to find out whether Hamburg was particularly bad or not By using all these tracking services and it's not transparent It's simply not easy to see whether, like, city websites do that or not So we are trying to increase the transparency by making all the data public By making all the results public so that everybody can see and make their own judgment about how the situation is So how do we actually do that? I need to rush, right? We have this architecture which sort of evolved a bit over time We have this, the queue of jobs which we distribute to several virtual machines We have around 30 right now, that includes, like, the web service, the publicly-facing web service And, like, 28 or so scanning, virtual scanning machines And that's basically just a Debian image which has Firefox and the open WPM and everything installed And then, well, it distributes jobs via a Redis queue and they return the results and everything And we do collect the data from all the hosts and then we interpret when rendering the results That's clever because then you can easily change your requirements and your ordering and everything And it's not so clever, though, because it's very intense in terms of computing So, just these three sides real quick I've mentioned that we put up the list of projects being presented here at POSDOM And these are some basic results So you might very well ask the question, how many cookies do I get when I visit eclipse.org? The answer is 78 And is that good or bad? Well, it's actually the worst This is like, you won't get as many cookies as from eclipse.org And is that normal? Well, it's not that unusual because LibreOffice.org gives you 75 Out of those, you get one third-party cookie Like, these are first-party cookies or second-party cookies These are third-party cookies and these are known trackers Like, companies that will collect your data over time and sell them Which companies who we know of that they do that And then you may ask, well, how worse is or how is the tracking situation? This is now sorted by trackers And we see that openset.org gives you five long-term tracking cookies of companies That we know will sell your profile after a certain while And the list is bad So here's the result, now you see how the situation is This is like a real quick, real quickly drawn table And you can see zero third-party requests are being performed by nine sites And then you can see how many requests are, how the distribution is of the third-party requests over sites And then I am finally able to close my session And I want to thank you for your attention And I hope I will see you soon on GitHub And I hope you will send more lists that will send bug reports, ideas and of course patches Thank you very much