 Who is excited for the last two-hour talk of the day? These guys have some awesome things they're going to show. And, and girl. Um, so, um, with no further delay, please give a warm welcome to Sven Till and Soggy, presenting at DEF CON 26 inside the fake science factory. Oh yeah, and if one falls over the other one because of the lav mics, um, I'll help them up, but, uh, yeah, let's, let's get you guys untangled first. Alright, give them another round of applause. Thanks for having us. This is amazing. This is such an honor to speak in front of such a great audience here. My name is Till, Till Kraus. I'm a reporter and investigative journalist from the Süddeutsche Zeitung, Germany, Süddeutsche Zeitung magazine, a big magazine from the German newspaper, and we're glad to be here today. My name, thanks. My name is Svea. I'm also an investigative reporter from Germany's biggest broadcaster, ARD. And the reason why, or one reason why we are here and why we are really excited to be here is that the whole research you're going to see now started right here at DEF CON. So this is for us also kind of a way of saying thank you DEF CON because of what this conference makes possible. And it started because last year I had a talk here and I met Chris. How about now? I had that same thing last year I should learn. I'm Chris, also answer to Suggy. I work for the online privacy foundation and I'm also a member of the DEF CON CFP review board. I didn't vote on my own talk. Honestly. So to get you started, I mean this is a really long talk but it's basically divided into two parts. So what we're going to do now is give you the presentation and have a little bit of time for questions and answers. And then you're in for a treat because we're going to go show you a documentary film that we made about the subject matter that we're now talking about. So you'll be well entertained I think, I hope. And after this documentary we will still have some time for questions and answers so if you guys want to know anything more that we're talking about we're here to answer your questions. And before we totally dive into this fake science factory what does it even mean, right? So we will talk today about something that pretends to be science but in fact it's just bogus or you could even call it bullshit sometimes. So in order to understand how important it is let's think a little bit about what science means for society, right? So when you talk about science it's not just academics working in ivory towers doing research the results really influence the world, how we see the world, our perception of the world what kind of things we buy, what medications we take how political decisions are influenced. A lot of this is based on science and this makes it really important that science is actually working the whole process of scientific publication. So about a year ago I was fairly far along with some research and with the research I wanted to present an abstract and some preliminary findings at a conference so I found this one, the 19th International Conference on Political Psychology submitted an abstract, it got accepted we were pretty delighted with that and that's a fairly standard thing to do in academia so that you can discuss your results with peers and that forms like the rest of your decision on how you write the rest of the paper. So after I went to Copenhagen in Denmark in October last year excited about the conference, kind of bricking it about the presentation and this was the conference. Not a room in the conference, that's the conference and all the attendees except me but I just thought well maybe that's like an admin error or something but then the talks started and bearing in mind I submitted on political psychology the talks before mine were on urban planning, advanced Islamic finance I'm not even sure what that really is, robotics, farming, all sorts of things you know get there, they say okay you've got five minutes to present five minutes, you know before they said it was 20 minutes so you get there it's five, it's like all of a sudden you're thinking this is a little bit weird, what the fuck? So this sort of shenanigans has actually got a name as I found out I'm the sort of schmuck that goes to these sort of places apparently and it's called predatory publishing and Till's going to talk to us a little bit more about that So predatory publishing, what is this? So we tried to nail it down to some sort of a formula based on Albert Einstein Euro is MC square so this means that what predatory publishers are doing is they take something that is either scientific like your talk or something that is utterly non-scientific they mix it all together to make a lot of money and there's a definition for this of course so this is kind of a broader sociological problem where you see that those meetings are set up to appear as if they are science but they're not because nobody's actually really looking at that what people are publishing there so my question to you guys would be who here is either scientist or has a scientific background and knows a little bit about that let's have a show of hands that's quite a few people but for the rest of us I'll just give you a very brief tour of how publications in the scientific world usually work so you'll in a good academic journal you have an idea for a paper, you submit it an editor checks it and could either reject it straight away or sends it to a process that is called peer review so peer review is something where other scientists other researchers who have some knowledge in that field look at your manuscript and make suggestions they look is the mythology okay is the data set okay are the conclusions in any way coherent and they read the paper, they make suggestions and they can again reject it straight away if they say nah this doesn't make any sense or they have it for revision and a friend of mine who's a scientist he calls the peer review a big pain in the ass basically because people always have some suggestions and it's a very very long and painful process it can take months so it goes back and forth you can have to resubmit it another peer review another possibility of rejection but then in the end after a long time it gets abstracted so a little disclaimer here the peer review process as we know it from the big journals that we know of either open access journals on the internet or printed journals it is not a perfect process sometimes weird papers slip through the companies who run those big publications there's a big monopoly here a lot of money to be made so this is not a perfect system but the whole idea of other scientists reviewing manuscripts still is kind of like the gold standard for academic publishing so when we look at those predatory journals things look quite a bit different so the only thing they have in common is there is a submission so somebody submits a paper there then you've got some superficial comments if at all and then it's accepted so quite easy right so in order to understand what we're talking about here we have to make clear there are hundreds if not thousands of predatory publishers out there they're a little bit like this Nigerian email scam that you guys probably know a lot about they send out emails in bulk trying to recruit authors people to send their manuscripts there and we looked at the five major companies behind that one of them is Wazid where you presented Omics is another one from India so those are the ones where a lot of journals are published they have a lot of conference going on so these are the major players in this field so after finding out about this whole predatory publishing thing I wanted two things wanted the 450 euros back and I mean the 450 euro thing is pretty standard if you've been to like an IEEE conference you have to pay for a tendency so it's like I didn't spot anything there but more importantly I wanted my intellectual property back if anybody's written an abstract it's kind of a pain in the ass and it takes a long time so having to rewrite it so it doesn't fall prey of some sort of plagiarism engine is a bit of a frustrating thing so I asked Wazid for my money back and them to withdraw the paper and they were like oh no you signed the forms here they are so I had to then go to the credit card company and explain what a predatory conference was and that's not that easy so you went to a conference yes but it wasn't really a conference right but there was a conference sort of so I had to write a detailed description for that and also to send to UK fraud authorities kind of explain what was going on so I could get my money back and I took all the information I'd written up and I just put it on a WordPress site wasitwatch.wordpress.com I also found all the authors that were going to be attending in the next 12 months and emailed all of those folks that were from universities and said hey you might be going to a shitty conference I also created a Twitter account wasitwatch and one people who were talking about was it and I also Ping Sveya here who I met last year at DEF CON and said this yes this is exactly the message I received and I would say don't mess with him because he knows me I'm badass on Saturdays and this is where the story begins with the whole investigation so we decided as a media organization to dive into it and to go on research this as deep as we could and to really kind of rip them apart so I teamed up with Till and also with another good colleague of mine which together we are quite a big investigative research corporation in Germany and one of the first things what we thought what we should do was need some help and I don't know who saw my talk last year knows that I love to dress up so so we needed some help and my alter ego was Isabella Stein and we decided to become scientists and Isabella Stein she was from a small university the University of Himmelpforten it's a village in Germany and I know a couple of hundreds of people living there and we believe that Santa Claus is coming from there so it was Isabella it was Isabella Stein from this fixtures university and also together with my good colleague we submitted a fake paper this paper was randomly generated by a computer algorithm so oh that's more I'm sure you all get that right this is a very neat program you can find it on the web it's called PsyGen and some MIT students invented this program you just have to type in your name your fixtures name and then a paper jumps out and this was the one when we typed in our name this was the paper which came out so fake paper number one is sting operation we decided to go to the same conference people where Chris was they call themselves the world academy of science and technology and we hardly believed ourselves when they invited us for the conference so we went to London this is not this one we went to London in January and presented there and yeah let's see how Isabella Stein and Christian Schreiberma how they did and we really we read it out loud let's see if we get that running yes the first one it's edited a little bit introduce us my colleague Isabella Stein also from from our university of applied sciences see here now is the relationship between our solution and the analysis of the memory bus this is memory bus and here on the bottom and here on the top of all of this there's a gauge where you would need a theory of rat like trees but we were pausing on that solution we used the 90s Nintendo Game Boy that's the solution which means the more pressure you give on the system the higher the scalability gets and that's what we wanted to achieve how would our system help us in here you all know this single particular is a reflection of quantum potentiality the three philosopher Plato and introduce us my colleague Isabella Stein also from sorry user error thank you guys the hardest part was really not to laugh and I have to admit after our little theater we nearly ran out the room in the next room and we couldn't stop it for a while so but I mean it's funny but on the other hand it's sad at once and everybody applauded and I think there were very polite people polite scientists but also there were no computer experts so there was probably a chemist or biologist or a film critical critics person so they couldn't know about what we were speaking and this is also a reason why these conferences are so oh I didn't want to spoil that one so this one is the organizer this was the only person there who was from the world academy of science and technology and I asked him who are you, what's your name and he barely wanted to answer and he mumbled something like I'm only a student from Cyprus and I don't know anything about this so we kind of stopped here in London because we could not find out who is that guy and who is behind was it and this was also a time when we needed some help more from a hacker person to die deeper so I called a friend of mine Andrew McPherson and Andrew McPherson is like employee number one of Poterva who make Maltigo I don't know if you've not heard of Maltigo it's kind of a tool for exploring relationships and technical and non-technical contexts and it's pretty awesome and the creator of Maltigo created this graphic for us because he was pretty excited by the talk too so in order to try and find out a little bit more about who's behind this wassit.org organization we plug this into Maltigo rather Andrew did on this occasion and we run a transform that shows entries from who is and we found out that wassit.org was using Cloudflare which is a little bit of a pain because it's a bit of a dead end but not really a pain because what we found with wassit.org is that it was using the same tracking cookie as is.org and wassit.com then if we look at the IP ownership for those guys we see it links to a guy called Bora Ardil and Bora Ardil is the guy and the student there the student there, the surveyor met in London Bora Ardil also posts on PHP Freaks under the name or the alias of Plobus and you can get some interesting information about the structure of the wassit organization by some of his posts on PHP Freak so back to Maltigo looking at the who is information we also see there that if we look at the is.org practice a million times it references a gentleman called Camel Ardil I'm not sure if I'm pronouncing that right but this is the guy here Camel is really the the top of the tree for wassit.org it's a family business he's Bora's father and there's another Ardil as well involved three of them we think if we look at Bora he's also registered a ton of other conference type DNS names 83 in total like conference university and scientific conference and stuff like that so in total host about 13 events in 13 different cities each month so they're quite busy 5,000 conference titles a month a year that's 157 events, 48 cities 35 countries and 53,467 conference titles in total anything you can think of and if it's not there there's an opportunity to suggest one so we estimated based on about 20 to 25 submissions per event that they're making or the annual revenue is about 3.8 million euros a year or about 4.5 million US dollars even if there are overheads for the hotel rooms and they don't necessarily need to book the hotel conference rooms all day my conference lasted two whole hours so even if it cost them 2 million that's quite a lot of revenue profit that they're making from this and this is just was it another video we of course tried to speak with them about this so we went to another conference in Berlin but this time not Isabella we have several questions to you is this science why would you call this a scientific conference so what is scientific about this conference was it of the IOC just let me call it you will face it you will face it general police this was the end of it so we got thrown out we got thrown out we didn't get any answers from them until today so the only answers or the only information we have or the information we have from Chris after this encounter with these weird people and these weird conferences we really thought let's look deeper into what's going on there we know that a lot of those conference organizers also have a publication arm where they publish journals and scientific articles so we thought ok if they accept any computer generated nonsense and you had a good laugh at that because you obviously were able to see what kind of crap that was let's see let's try to put this on the next level let's try to have a scientific article published in one of their journals so we really try to think what would the bad guys do right so we try to have a we invented a cancer cure complete nonsense and had the goal to have it published in one of their journals so that we could say well this is a scientifically proven medication that we could then sell for example over the internet because who would buy just some random medication on the internet when you can buy one that says this is proven by scientists in a peer reviewed journal so what we did we for some reason we like bees because bees are awesome can bees heal cancer? probably not but who knows in the world of scientific publications that are fake anything's possible so we invented another institution the ephabia institute we made a twitter account and a website a twitter account logo actually is the german symbol for recycling so this was a little hint for the trash that we're going to put out to the world the CEO of this is Dr. Richard Funden which in Germany means erfunden which means invented so the whole thing was pretty bad and we submitted a paper to one of the journals from omics which is one of the biggest companies in that field the journal of integrative oncology looks legit at a very first glance so we wrote a paper saying that the wax that bees produce can have a better effect on cancer treatment than chemotherapy so we just we just said that right there was no proof we said oh yeah we had some patients and we basically asked them as long as we wanted until they actually said yeah we feel better this was our mythology and we made some other outrageous claims for example we said well one of the reasons why bees may be a good you know therapy option for cancer patients is because bees don't get cancer right makes perfect sense the other argument we had was that well the general nature of bees you know how they fly around and those are really happy animals that pretty much suggests that they're probably a good way to treat cancer and we even because we're scientists we quoted a book you know we just didn't say that bees have a very very happy nature we looked it up in a book and we quoted it in reference section the only problem is it's a children's book we really we really try to make it low-stakes we're not like the a Nobel Prize winning scientist is needed to find out that this is utterly nonsense right but guess what happened after a couple of days we got review comments back from the publisher and they said oh yeah this provides important experimental and preclinical evidences there was no evidence at all it was just completely invented they had some minor corrections that they didn't understand and abbreviation what SITF means because we just made it up so we said oh yeah SITF stands for signal infer transfer protocol and they said oh sure yeah okay and we got an email around ten days after we handed in the manuscript and the paper was accepted and published and they wanted to have two thousand euros so two point something thousand dollars publication fee which we never paid and they still published the paper so it was online there and it doesn't stop there so after a while when the paper was online we received several emails from other predatory publishers inviting us to be editors at cancer journals and one email that I put up on the screen here we got invited to be keynote speakers at a breast cancer conference in Paris so what a career right we just invented this whole operation and within two weeks we had a scientific publication that we could put on the website and saying this is officially peer reviewed and we were able to speak at a conference and we're editors of journals so if we really had the plan to sell this fake medication we would have some arguments now and this sounds funny at first and believe us we had a lot of fun doing this but it's actually really serious because this is not something that we just made up this is something that is really happening so there are various medications or alleged medications out there which are proven in these journals for example it did not took us a long time to find GCMAP GCMAP is supposed to help against last stage cancer and a variety of other other illnesses you may have and it is something that you take it and then your immune system gets so strong that cancer is beaten and they advertise with these studies really quite often so we just want to show or give you a short glimpse of what you can do with these studies it's particularly relevant for the drug oxyhalin 300 scientists from 8 nations have written 150 scientific research papers on GCMAP 200 scientists but we have written 32 of them yes this is advertisement go on let it play and you will see now testimonials from this company I took the GCMAP twice a week and after 3 weeks I started to feel less tired it isn't something that's just quackery it is scientifically backed so we will now have a short look into the studies 300 scientists from 8 nations have written 150 scientific research papers on GCMAP 200 scientists but we have written 32 of them he wants to stay with us I took the GCMAP twice a week and after 3 weeks I started to feel less tired it isn't something that's you know just quackery it is scientifically backed hey there we go it's a good advertisement 300 scientists from 8 nations make it stop but I think you are getting the idea right this is what makes this whole fake science operation so dangerous that people can actually sit in front of a camera and say this is not just quackery it is scientifically backed let's be honest who really has the background to double check those studies yes and here you see some of the studies from this company and these studies they are one of them is in the journal where Till submitted the B-Wax paper others are in journals where we submitted a computer generated fake paper so you have no peer review or you have some kind of fake peer review because this is the only reason why these papers get published so what we did was we showed these studies to a well-known oncologist in Germany and she reviewed these studies for a second time and she said these studies are really terrible and that she thinks that a normal person can't see it and even a doctor if he is not very familiar with it even a doctor can't see it so we take care about single cases for example they are not scientifically at all and they should not have been published in any real journal so her conclusion was that these studies only exist that these internet pages where these products are sold can link to these scientific articles and this is quite bad especially for the patients when they are very very ill they are searching hopelessly for some miracle or for some cure and this is also the way how these studies are spreading over social media or over other media over articles in media so it also took us not long to find somebody who also spread the word this is a very beloved TV host from Germany she died in 2016 because she was severely ill she had breast cancer in the last stage and she wrote a book and in this book she really spoke very advertising and well about GCMF and that this is her last hope we also spoke to her best friend and she told us that this was her last hope this medication and she was nobody who believed in some wonder healer or something she believed in the studies because she read them by herself so she died and on this case you really can see that this is a business with last hopes and that these people who are publishing these failed studies that they are making money with the hopes of dying people so usually there are no consequences in this case luckily there are so the one company who is selling that stuff is going to go on trial in London in November and the files alleged that they illegally sold GCMF as a cancer drug based on failed studies we also try to reach out to them but our questions were ignored so what's the matter with all this many snake oil sellers they can use this and then they can sell their stuff we found plenty of other medications like some very themselves therapies which can't be working or some a bio energy healer who has 150 studies who heals with his energy so there's plenty of them out there and they can sell their products because of these predatory publishers so this was another reason for us to build a bigger picture we wanted to know who else is there and the first step was to write as many fake papers as we could and get them accepted I think in the end 12 different publisher this one was mine we've got a real scientist here right thank you but to be honest two or three papers did not get accepted so that works as well because this is meaningless do something do something better in the next future so we felt some kind of didn't accept it but the absolute majority accepted the paper without any comments after we asked them this publisher why did you why did you accept this or your predatory most of them said no comment some said no we are not predatory publisher even if they accepted two papers from us like the most they said we are a platform so we are not responsible but we wanted to go and to dig deeper yeah so to gather some information from the various predatory journals we did some scraping and analysing or spidering scraping and analysing one of those and we start off with scraping which we divide into two sections we want to look at the abstract and papers that have been submitted and all of the information that goes with those and the other thing we wanted to do was look at all of the conferences and where they are and how frequently so the first step we did is I do some sort of site recon and in this example we just look at waslet real quick we get some ideas of the site layout and what we want to set our spiders to and what we want to scrape and then later pass so you see the papers and the abstracts are listed here and we go into more detail and here you get things like the author name, the journal title things like that and also you get this like unique identifier for the abstract and they range from about zero to 100,000 so you can write some or I wrote some messy python to just loop through all of those and download the html files of which there are 53,069 for was it and then pulled out the metadata the title, author, date, journal name stuff like that and piped that all to a CSV file there's no real magic there it was quite straightforward the resultant abstract CSV file had all of the things we just saw, the paper ID had the author stuff like that and a direct download link to the pdf which would either be the pdf version of the abstracts or the paper and those pdfs contained more information such as the institutions authors were at and their email addresses so we pulled down the pdfs used pdf to text and piped some of that information into abstracts.csv as well others on the team preferred a slightly more elegant approach using scrapey for example where you're really doing your sort of spidering and parsing all in one and you're getting your sort of JSON file out of it next moving on to conferences real quick there are 50 different sort of conference areas within was it and each of them linked to hundreds together collectively thousands of conference titles so 50 subject areas and here we see one subject area and that's just a small snippet of the different conferences in that subject area in August 2018 and the reason why they all have those abbreviations you know you see like ICI something that's done on purpose because legitimate scientific conferences usually go by those acronyms so what was it did is they just changed one of the letters around so that it almost sounded like the original conference but not quite and so this tricked people into believing oh I'm going to reputable conference whereas it was just you know as valuable as a Gucci bag written with one C so you could have been at this conference but you chose to come to DEF CON now just finished in Vancouver and apparently they had scenes when a lot of the attendees the penny dropped for them so there was a bit of a backlash there apparently so all together there are 44,476 different conference titles you can select there's going to be one for you and if you're the kind of person that likes to plan in advance DT take note and look at all of the wonderful cities you could travel to you'll get a receipt and an attendance voucher and all of that sort of stuff 2031 so we take what we did there with WASIT but we applied that to one of the five Predatory publishers that we looked at we get the JSON files, the CSV files and we use a collection of tools to do various different analysis depending on who we were and what we were looking for so Excel Tableau which is you can get 15 day free license or trial license for that Neo4j for graph visualization, linkurious and of course like R and Python as well so on to analyzing the data that we downloaded yes it felt like like lowering a curtain like looking behind the scene because after the scraping process we could finally say okay this is how big you are this is how many abstracts you have so we wanted to know how many authors so we found nearly 180,000 abstracts and around 400,000 authors contributing to this scam worldwide and this maybe some people would say oh my god science and danger so no so this is if you compared to the total number of scientists which are nearly 8 million scientists worldwide this is still a very small proportion so most of the scientists are publishing in regular journals, open access journals or paper journals and only a very small proportion fell for this scam but the development is quite interesting you see here that especially with the big ones omics and that they have had quite an increase especially in the last 3 to 5 years they are getting bigger and bigger making more money and many people out there are saying that this is a problem from lower income countries no it's not the US is the second biggest contributor to this conference especially omics with nearly 10,000 abstracts and what is interesting if you dive deeper into the data when you are searching for universities or when you are looking for who is contributing to them this goes in every field really no university you won't find there so even elite universities have published there over the past 10 years I have to admit so probably one can say ok this is not too much but anyway the numbers we pulled out of the metadata from the paper I also did here for DEF CON especially top US institutions list also out of interest because I wanted to know which universities are on top of this list it is an institution it's the Mayo Clinic I think it's pretty well known institution University of Michigan Wayne State University I wrote all of the universities you can read here I wrote all of them email asked for their comment most of them did not comment on the issue some did so one here to read so they are really dismayed about this they were really kind of concerned when I talked with them on the phone they were like oh my god we did not know this we don't like this that our scientists are publishing there and they want to take care and they want to take action to stop this and to inform their people about this so most interesting is what are the reasons why scientists do this so first of all scientists got scammed we were talking about the Nigeria scam so this is they are sending spam emails and you fell for it accidentally then you go there once and then you see okay this is crap you won't go there twice also there's the publish or perish pressure and academia so probably some people choose the easy way because it's just fast to publish and the third case I want to go a little bit deeper because this is for us it was the most interesting case when scientists are taking advantage of this predatory publishing so we were lucky to find one case where there is an investigation ongoing so this is from CUNY University of New York and there are quite some professors who really like predatory publishers so they don't publish their once or twice no no no no they publish their quite many times so what you find in these papers when you print them out you find that there's some kind of authors filing you find that 9 to 15 authors for one paper which is not normal then all of these papers are brand funded this usually tax payers money which is in these papers then a lot of these papers is copy paste so if you run this through a software you really see that there's a lot of copy paste going on and if you take them to a scientist to do a peer review what we did then they told us okay some of this what is in there is really scientifically questionable because probably they were written in a very short time we were lucky to have had a whistleblower who was familiar with this case with this CUNY case and he he told us that professors can benefit from gaining higher salaries at promotions because they probably have some legit papers from good journals and then they can just fill it up with this cheap publications they can also can obtain other benefits and this was the reason why this is or this could be the reason why this is so liked for some for minority of people which we find interesting is that even if you do this knowingly or unknowingly it doesn't really matter because you are feeding the system with this the predatory publishers they get the money from the universities so this is one thing and so they can grow with their business model and the other thing is they get the reputation of the universities they can advertise with people from Stanford are publishing here or people from Harvard are publishing here so even if the scientists don't know so even if they do this unknowingly they are helping the predatory publishing and they are helping a different group to know more about so when we looked at this we thought well this goes way beyond academia right so it's one thing if professors try to polish their publication lists and get more money maybe or have a better reputation but this whole business can really be used for all kinds of purposes because we as a society we still have this kind of feeling that if something is scientifically proven it's kind of valuable and this is great that this is the case because usually science does exactly that but in the case of the predatory it is quite different so we not only looked what academics are doing there but we looked what our companies and lobby groups and political influencers are doing there because in the end big companies and big corporations have research and development departments where they present their own research and their new products and all these kind of things and oftentimes what they find is not scientifically sound but good enough for the predators right so one big branch of course the tobacco industry they have a reputation and I think most of you guys you know about this they have a reputation for deceiving the public for decades trying to downplay the danger of secondhand smoking or the dangers of smoking altogether you probably know that spiel right it's like oh well cancer has many reasons and smoking maybe one and maybe not I don't know let's leave that's kind of what they do sometimes and so what they do now particularly Philip Morris is a company that we found there quite often producer of Marlboro you probably know them so they start to revamp their business a little bit by selling e-cigarettes that are potentially less harmful as they claim they have scientists writing about this and where does the science sometimes end in the predatory publishers so they have those well looking brochures that they hand out to investors and to the public where they make their health claims and they list the articles here as peer reviewed and there are from Wasett they were published in Wasett and we know what Wasett is right we got a best presentation award for reading a computer generated nonsense paper there so well not quite reputable I guess so but this is not a one time shot Philip Morris seems to be quite a good customer for them so they publish all kinds of studies in those non peer reviewed journals that claim to have a peer review they go to conferences present their research there and particularly for the tobacco industry it's interesting because for some serious academic conferences they are banned they're not even allowed to go there because of their history of deceiving the public they're on a blacklist so they just it seems that they found another way to disseminate their research to kind of boost their reputation who else is there? the pharmaceutical companies the pharma giant buyer from Germany inventor of aspirin they published there quite a few papers and one of them is actually interesting because it actually has to do with aspirin their flagship product because they want to sell different variations of aspirin they come up with aspirin plus C which is basically just normal aspirin with vitamin plus C at it they have a paper here claiming that this helps against the common cold that we showed the paper say well this is not actually a legitimate claim the German consumer protection agency actually says that the addition of vitamin C doesn't make any sense in this case but you can sell it at a higher price so if you are a consumer and you want to find out oh well there's this normal aspirin and there's this aspirin plus C you probably go to the internet look up what the benefits are and you end up here so if you google aspirin plus C the omics paper and let's keep in mind this is non peer review this is just any you know any study you can have published there is on the second rank on google so I think now with you guys actually looking it up and clicking on the link you may actually make it rise to the top so again that's the collateral damage of a talk like this but well let's give this to Bayer they can have it okay so I think you understand what's going on here right other companies Malincrow controversial company that's just recently been fined a hundred million dollars for absurd and obscene of the prices for the medications they published there about medications their AstraZeneca is doing it lobby groups Ilse Europe a think tank funded by Coca-Cola Hershey's and Kellogg's they sent their scientists over to a conference speaking about childhood obesity and nutrition and guess what it was not about salad so they used this kind of forum to disseminate their research what else did we found we were kind of a run right we were typing in all the companies so we found critical infrastructure from a company that's responsible for the nuclear safety they published things there and was it again you know the the company that we just saw and it just goes on and on institutions from Germany basically tax money goes up they present their stuff there and once you publish the scientific publication it just doesn't end there you know so it it's cited in other publications other people you know cited it's cited in patents for example for somebody making patents on medical products they cite was it publications there and what we found particularly disturbing there's this whole big group of climate change deniers the co2 coalition for example here in the United States very very controversial people you've got the IK Institute in Germany that is scientifically working together with the German right-wing party AFD they speak in front of parliaments in Germany actually presenting their view that climate change is not man made this is all not a problem and we should all you know further invest in coal and all these kind of things they use studies in those journals to back up their claims so this is a very common strategy in scientific political propaganda that you say well look on the one hand you've got all those award-winning scientists those great you know thinkers who come up with their theories and their proofs but we've got other studies that just say a different you know that just say the opposite and in this case we could really prove that their arguments come from predatory publishers and how do we know well we just actually submitted a computer generated nonsense paper at the exact same journal and got accepted like this in order to kind of conclude this first part we really have to think again what does science mean and we got back to the slide from the beginning so this is not just about your common professor doing a little bit of research on the side scientific progress is really a super important driver for our society in the age of enlightenment and democracy so when scientists find out influences not only political decisions but what we buy we think and how we see the world and if you know with the knowledge that you all have now about the predatory publishers I think you see the danger that is in place there and anything can be disseminated anything can have the aura of science nobody checks it and when we confronted many universities they were absolutely clueless they have never heard of this problem before and so the societies those studies they're spreading like viruses you know there are a lot of dangerous things are happening you can in the end no longer distinguish fiction from fact so the most terrible thing that could happen with this whole thing is that the trust that we have in science erodes because we think science is awesome we think science is great and when they do this thing they really sell themselves at a very very cheap price and the trust that we have in science and progress may erode you know if more people publish in those journals the trust is gone and this would be really terrible so in order to kind of combat this we really made a long streak of investigative projects where we published the findings that we had together with 23 media partners and journalists around the world where we shared the data and people from other countries be it Korea or India or the United States or Austria France they looked into specific cases from their countries which universities are there which companies are there Germany from the 30 biggest companies 12 had publications in those journals and this this big kind of publication really had the aim to raise awareness as I said don't mess with him yeah it's all because of course yeah or of that kind right so those are some of the results of the publications that came out and there's more to come the Guardian just ran a story today and there's more coming up for this in the future so never end a talk without a call to action that's what we learned right so we're speaking in front of a very very curious and interested audience here so I think what you all can do is you can actually help to make this problem go away when you find a study that somebody some wacky person cites and says oh you know what autism doesn't exist or all you know tobacco is not harmful look where this published look where this study has been published and if you find it to be from a predatory publisher say so and share the share the word and spread the word about those companies if you look up your university or if you have academic friends warn them tell them that this is actually hurting science by publishing in those journals and again we know that academic publishing has its flaws and even the established publishers do some mistakes but what those predators are doing is really hurting everyone so the big point is here you all can help with this project because you don't even need the database and all those files this is just if you want to dive deeper but if you have a simple google string search that Svea is just about to tweet out on her account if you enter any word there be it a university email address that you know of be it a controversial product that you have heard of be it a controversial person or anything that you want if you google it through this chances are really high that you will find if this comes from a predatory publisher or not look for government we found a lot of government in Germany so look for surveillance because this would be very interesting if some surveillance stuff is sold with this with these studies yeah the military is there sometimes too right because all those people they need publications they need the aura of science and this is the cheap and easy way to do it oh yeah there was a dot gov that just submitted to the Vancouver conference just been so you could look for that yeah so I think this concludes the first part of our presentation there's of course this was a big research and a big work from a lot of people so this whole project is more than the three people here so we thank all the people on this chart and invite you to stick around for the documentary that we're just about to show and yeah we are approachable on twitter if you find something get in touch with us we'll hang around after the show here of course come talk to us we share our knowledge and we thank you very very much for the attention this has been great this has been a pleasure Nefcon you guys rock we'll get the we'll get the documentary we'll get the documentary set up those guys would have just given me my money back when I asked you know that rug really tied the room together yeah and so the documentary that we're showing now of course is subtitles so you will understand it and if you have any questions now while we're setting up the laptop we're happy to answer if there's a quick question we have some time so come come in front or speak it out I think there's a question here of the legitimate well the question was from the papers that we looked up how many of them were really legitimate that is a very great question and the answer is it's hard to tell because only some of them that we really found out were seemingly really dangerous for example with the cancer medications we gave them to other legitimate scientists to look them up so we didn't of course control all of the papers and see how good they are but we really think the problem is that it is almost impossible to tell the difference so we kind of had the comparison like in Germany you have like a big institution that every car has to go there I think it's like the DMV here or something right you get stickers on your car that checks if everything is okay those predatory publishers are like the DMV just putting the stickers on any car without looking at them most cars will probably be okay and we'll have exhumes that are poisonous and some of them won't have breaks because nobody looked at them and this is the problem with those journals that nobody looks at them so the filtering function is gone now everything is science and everything is peer reviewed and you can't tell the difference so we invite everyone here to find papers and go for a hunt and look it up and if you have expert friends and you find a paper show it to them and share the knowledge and if authors are publishing papers to these places and kind of leaving them there granted sometimes it's hard to get your paper removed so that's a factor but if people are publishing there which is kind of deceptive in its own right then you have to then wonder is the whole pool polluted how was the research methodology in the data we just don't know and when there are open access and was it prepub paper you know sites like archive and osf.io you have to question why researchers aren't publishing there instead so it's really difficult to sort of extract what's real and what's not and what were the people's motivations for publishing there we've got another question there so the question was how those business models of the conferences and the publications are intertwined and the answer is yes they are absolutely intertwined for example omics one of the key players in this they offer both they do a lot of conferences and they do a lot of journals and there's currently actually an FTC investigation here in the United States against omics because of their deceptive practices and so yes a lot of people who publish there also go to the conferences of course there's some separations as well some people only go to conferences others only publish but many do both full package right another question yeah yeah we actually we researched one case in Germany where we had a professor and in Germany it is like you have to collect papers and if you have enough paper then you are a professor and in this case we found his work and a lot of his work was from predatory publishers and he was a professor he was actually a global professor because he was very young I think he actually had two or three PhD degrees and he was very young professor and yeah we called him he sweated a lot we did a lot of phone calls we didn't publish his name but if the people don't look closely they may not see that it is published in predatory journals what's his name poor guy yeah there was also I'm sorry question over there thanks we were really often asked for doing this the question was are we going to make the database available but we can't do this because it's a legal problem because in Germany it is when you publish a person's name then you first have to give him the opportunity to say something for these allegations and with 400,000 scientists this would not be possible for us so the only thing what we can do and this is what we did here is to publish this easy Google search so that everybody can track his institution and we tried to help people to do the scraping or probably to do a scraping project put it on github so that everybody can do it because we cannot publish the whole database in the web yeah, go on please great question, yes that is a good question so the question was if we came across the fact that some of those journals had people there as editors or editorial board members and yes we came across tons of people who were just there either who were already dead that happens so not a lot of editorial work from their side or people who are absolutely clueless about this because you know those website photos they're public, the CVs are public so those predatory publishers just take the photos on the website, put them up there and when we asked them we wrote emails to those people who were publishers there or an editor there and a lot of people replied I have never heard of this and they tried now legal steps to get their pictures removed but we don't know how this will end but with the scraping the scraping code only took the papers and the abstracts scraping code did not include the editors but what the scraping code of course included was paper who probably where people wanted to retract it but didn't get through it so yes of course and the 400,000 scientists there there might be a quota of people who is in there but who definitely not want to be in there and there's a question over so maybe let's take one or two more questions we will have more questions after the document one more question, the guy has been standing patiently with his arm, you, yeah you mean like the GC Math case yeah, while we're investigating still into the data, yeah there's more stuff coming up there but we really hope from input from other people because some of those findings you just have to come across them, you know we don't know about any wonder drug that's sold in Minnesota or South Korea or wherever around the world those snake or sellers do their business so yeah, of course we still keep looking into these data sets and see what we can find but since it's kind of easy to find those products we really hope that other people pick it up and do the work as well yeah, the Google search will get you quite far actually, and the scraping is super easy too should we roll the video? yeah, so let's watch the film it's like around 30 minutes and we'll stick around and take more questions afterwards thanks so far, this has been great at some point, in large numbers we have won the war we have won the war she also told me about studies she's read and I still know how much hope I had when she came that she found something new and that was like a straw home what happens when scientific studies spread around the network that's a film about the business with doubtful science and the threat to us the case of the moderator Miriam Pielhau he shows me how we can be manipulated by certain evidence her fear was exploited it's very emotional cancer makes fear cancer makes fear the moderators had doubtful studies a great hope this kind of studies spreads quickly I wonder who we can still trust Miriam Pielhau researches unharmed about her illness before her death she writes to her friend from a new scientific evidence the name G.C. Muff in an e-mail it says I pray and beg but I also have hope Google G.C. Muff if you have time it will bring me the final healing her best friend remembers she said I discovered a substance in the body that activates good cells and the cancer cells will be destroyed a week later she writes a shock there is still chemo in the blue maybe it doesn't work but I don't take any more I have decided instead G.C. Muff high dosage the manufacturers claim G.C. Muff aggressive as scientifically tested healing it is not allowed in Germany as medicine 4 months before her death a short vacation with friends Miriam Pielhau takes the medicine regularly she trusts it no one who just falls into shalatane Miri was an expert she knew a lot she worked with all studies and that was a big hope G.C. Muff costs a lot money a business with hope in July 2016 at Krebs the science and its studies builds our society we are quite proud of the knowledge of the researchers a strict process determines whether something should be proven it is internationally recognized and always the same when a scientist has written a study he sends it to a scientific journal experienced scientists check the study this is how it is sorted out mistakes correct control by colleagues peer review is the process we put an experienced Krebs doctor the study to G.C. Muff for a test she says there is a world in which the strict rules of peer reviews do not apply as a reviewer you would have high doubts about the data and would say it has to be written again then you would check it if it is correct the expert finds many mistakes the studies only look at individual cases use unsuitable research methods are full of positive reviews they are so lacking that they could never appear in a scientific journal with the studies the manufacturer of G.C. Muff at Krebs congress becomes manipulative 300 scientists from 8 nations wrote 180 studies to G.C. Muff and we have found 32 of them but we have written 32 of them I took G.C. Muff twice a week and after 3 weeks I felt less tired this is not a nonsense it all has a scientific background I think for a line it is not transparent and I think why these scientific publications do not mean to convince the professional because we can see what the quality is but from the internet where the line is explained where you make a sales offer then on these scientific articles how can it be that such studies can simply go around my colleague and I find out do us scientists a computer program creates a study out of complete nonsense we have only given the sales the university of Himmelpforten which does not exist here it should also be the post office of the Christmas man we reach the paper where the studies to G.C. Muff are available to be a scientific publication it is called Science Publications and lists 33 different journals with more than 10,000 studies as well as SciPub there are many serious pseudoverlages they are called omics Science Domain they do not work or react it is an industry that is spread out with which a lot of money is earned will our paper go through? Gerd Antes has been testing more than 25 years systematic medical studies is a scientific evidence of the renowned Cochrane-Deutsch-Land-Stiftung that someone says can go through does not come and if it comes it is a very big alarm sign that someone was either sloppy or too large or too large unbelievable SciPub has accepted our quads and we want to go on because the fake science branch also establishes so-called professional conferences here too we reach the emergency for an international conference in London at the World Academy of Science and Technology Vaset there are all three days of a conference there is also a peer review the test by colleagues but our quads paper went through we should keep the contract last test before the entrance really safe we are not yet together with my colleague Isabella Stein also from the University of Applied Sciences of Law Saxony at Himmelt-Vorten and the next morning we stand up as a false scientist before it stands out the great sound world academy of science organizes its conference in a small conference room and only has a single employee what a disappointment and this employee Bora Adil is at the same time the son of the business leader our non-sense study is already there printed in the conference journal without changes so obviously without peer review otherwise it would be our stupidity immediately noticed everyone is allowed to talk on mathematics follows engineering science and Korean history then film critic many scientists are under pressure they have to publish to get money a serious conference makes a good impression I do a project and I have to publish I only have to have what looks like a good publication what has a good name and if we present our quads how will the scientists react to this then it starts we stand up as Isabel Einstein and Christian Schreiberma from the University of Himmelpfort with us it is about a computer program with the well-known name MOP this is what we thought allegedly also works on a gameboy the total nonsense now not loud finally a wrong Plato quote no one notices it is simply no computer expert there without peer review for publication the simple way that's why they are here tell us some scientists others fell into one of the well-known conference names honestly I'm just shocked I was very early this morning but I didn't see only one guy who was still building and cleaning that was all very unusual what do you think the makers of the conference are doing I suppose they are acting that's the only purpose of this event the makers are making fun of the scientists that is respectful with research and if scientists publish here more often then it is for me the credibility of their publications completely in question around 23,000 euros this conference was brought a good business for Bohrer Adil we will meet him again later was it the world academy of science and technology just threatened or even more a washing machine a platform for bad science for lobbyists and business makers of their theories then in society as seriously presented truth they also find a move into the political everyday life one only in the fantasy greener ideologues existing crisis I speak of climate change because the climate change is not made by people the AFD has the right to demand back to the scientific foundation of the German and international climate policy he is part of the scientific foundation the name is Limburg Michael Limburg AFD member and guest today as an expert in Brandenburger Landtag Michael Limburg is vice president of EICL an institute for energy and climate development for what? for EICL Michael Limburg and numerous other scientists their studies which also have a peer review they present seriously but also an article those are the three things which we can work with for the AFD institute it is of great importance they win with credibility we are the age of enlightenment and science gives us the possibility to go on the basis and to separate from science Landtag session in Brandenburg Michael Limburg was in charge of wind power the use of wind energy to protect the affected citizens damage which causes a major damage damage to the health of the affected citizens which is caused by the continuous infrasight by these industrial monsters Michael Limburg is in charge of studies which want to make the climate change a natural phenomenon no evidence no evidence for the fact that the human atmosphere warms the planet in some mysterious way but we can find that some of the climate studies have been published by every quads I am getting a new fantasy study with exactly the same textbook which a scientist published our nonsense is going on exactly the same as the eike study our is the complete nonsense and the studies of eike at least doubt we would have found and Michael Limburg no one has Einstein's relativity theory what should I say we have to try to evaluate without having to deal with the content studies with peer review especially on critical issues they can manipulate us the damage is also dramatic when you think about the climate change or if you want to call it a world-threatening development when you think about drawing a mouse what does it mean if you want to come out then it is obvious that the damage is maximal the AFD is obviously doing no research on demand no answer we also find companies like the Tabak company Philip Morris it is being published on demand a company writes about safety the farmer in Sebaia to Asperin plus C BMW and Airbus are also published here the studies are good, middle or wrong no one knows no one has checked it on demand companies are shocked the problem is unknown unneeded studies they are also in documents of the European Commission in patent contracts for medical products and in the database it decides whether a medicine will be paid by the patient on demand we secure the GBA the studies were decided by the assessment of the medicines New York here is a Vaset conference city again my colleague and I want to meet a scientist from Hannover whose institute has been going to such conferences for years what do scientists drive that they present here to spread the fake tax we meet a professor from Brazil who looks at the business model this is a non-scientific it has nothing to do with science it is a commercial institution that makes money with science to make money then it is there the scientist from Hannover he comes with his friend two hours after the conference officially started he has been a division leader for five years at the institute with the money of his institute he supports the business of the fake tax I find it highly irresponsible to publish from serious scientists where we can see serious authors who are given space to do so why is the scientist from Hannover still here why is the public money to fly to New York for a fake conference he does not want to answer in front of the camera he only says New York is a beautiful city and he is now the last one of his institute in the future the institute for factory systems and logistics at the university of Hannover a responsible person has made an interview Peter Niehus is the head of the department he provides the government in the German Science Council and works on millions of research projects 29 times his name was published on Vaset New York one answer was New York is a beautiful city yes, these are public money how do you explain or how do you justify it these are tax money the organization has learned I think this process is described you can look at it it is not like we do not react the question is how long it takes until you really get it yes, of course, these are tax money or these are money from the seller but these are also tax money which are used there the conclusion is the one we have decided you can no longer go is it a bit of a disappointment if I write a paper as a review for example on my publication but it did not find a review there, colleague no doubt we got something and at the moment when it is given where we also communicate as a review paper we actually get something as well where I have to say not scientifically that we are in such a system and the scientist from Hannover who we met in New York the institute is not directly exposed to him we scientists have still published fake publications we connect with other AID journalists from the South German newspaper magazine and international partners 170,000 publications are published from different fake publications it is a gigantic cosmos there are around 400,000 scientists even from Harvard and Stanford and hundreds of German universities and research institutions down there the Charité Berlin the TU Munich the Fraunhofer Society the RWTH Aachen and the Hannover University more than 5,000 German scientists are published alone at 3 of the large fake publications all of us who write per e-mail they didn't know about our curiosity it is a single case and you want to address the problem that we have enough black sheep around that is clear but the extent what I have seen on their teeth that is amazing especially because it goes into all areas Lindau at Bodensee annual Nobel Prize winners many scientists tell us the pressure to publish is enormous a dramatic development of the last years and that would use the fake publications to offer an easy way Stefan Hell 2014 Nobel Prize winner he reports daily he would use e-mails from fake publications to use these easy ways if it has a system and people don't just use it and then you have to remove it and you can think how to remove it he sees the science in the duty but most of them are unknown Robert Huber, Nobel Prize winner of 1988 this is the fake publication we show him our fake studies reports of the research he was invited to a conference of fake publications he didn't think about it the serious photo of his lecture uses the offer as advertising until we report it he didn't know the event itself was criminal but he finds the business model of this publication he has to be followed that is clear there is no need to go out and search for authors they are all fair but then also what he promises if he doesn't do it he continues a conference in Berlin Bora Adil costs 450€ in the year he makes 3 million euros we tried to find more about the Turkish family but there is not an official business they already asked but no answer now we want to ask is the science is the science let me call you the love year just wait you will face it Bora Adil finds himself he says he is only a small employee in the end no answer there are few consequences for the events of fake conferences or the makers of fake studies the British manufacturers are an exception in November they will be made in London the business leader always presented himself as a cleaner in the files he would have sold fake medicine on the basis of fake studies also on our request no answer but the fake studies their probably scientific promises the official science about GCMAF is spreading the Miriam Pielhau about her illness I think it would be in Miri's mind if she had experienced it it would be the same and I think it is important to clarify fake studies a business for the makers a danger for the society who can stop it the lack of research is called the Ministry of Education but in the end the responsibility lies with the scientists I am really curious what will happen I have two tips one is the scientific community will do everything to let it disappear I fear we will do number 2 to see if the existence is destroyed I would rather not think it is possible it is difficult to live in a world where you can no longer distinguish between what is proven and what is fake who should you believe and what if you can no longer believe I have to figure out how to stop this it starts again I am trapped in some loophole time continues thanks for sticking around and for watching even it was all in German we shared the link on Twitter so if you want to share the video it is on my sessions website also subtitled spread the word I think if nobody goes there if nobody publishes there this is a pretty easy way to end their business model thank you we are happy to take a few questions while we clear up before we get thrown out there was a couple of points we haven't figured out where all of the money went you know for the conferences what else does that fund so there are some interesting angles to explore and we will be digging a little bit deeper and if anybody asked themselves what actually happened to our terrible cancer paper we thought we don't want this to be out in the public anymore we just try to prove a point here so what we did we wrote an email to omics saying oh you know what we made some terrible mistakes and by terrible mistakes we mean everything the methodology is crappy we send them a whole list of stuff that is completely wrong and ask them to take the paper of line so guess what happened they wanted to charge us we can take it offline but we need a retraction fee whatever that is for 2000 euros so we didn't pay but the moment we kind of said that we were journalists investigating their predatory publishing model the same day the thing was gone so it's no longer there luckily but we had to really put them under pressure with journalistic research because if we were just normal people we would have to pay 2000 euros to get the stuff removed so they earn money both ends looks like there is a question over there might not be able to hear you may need to come a little bit further forward but you can shout give it a go no I wish we did but there was a lady that tweeted the was it watch website yesterday and said oh you should have seen the scenes in vancouver when people found out this was like a bullshit factory it might be interesting to follow up with her and see what was going on but there was also a us.gov paper I think that was submitted to that conference not the cyber security conference but one of the conferences out of the 150 odd conferences held in that small room there's a another question there yeah you sorry I mean that's vague but yeah so the question is if I guess if papers have been published there maybe some PhDs might be provoked what do we think about that not provoked what's the word for it revoked yeah it's been a long day we actually can't tell maybe I mean all what we could do is spread the word and make this whole thing public and and and we confronted many many universities with our findings and now it's up to them what they do and probably they will have to look into the specific papers whether or not they are worth of the scientific label or not so I think it's up to the universities in the end please you say it's up to the universities is there no international or national level clearing house or agency who's supposed to be doing this job that you guys have been doing who's supposed to be doing the job that you guys have been doing and I guess holding these folks to account right well to our knowledge there is no like national or international entity that controls the publication process of scientists so I think there needs to be more like this you know it's hard to make this there's a freedom of publication for scientists and I think this is a very important good that you can publish your study wherever you want first the problem is only if you publish your study and you write peer reviewed and scientifically on it then it you know what's on it should be in it and this is with this predatory publisher which is not so the universities I think there is some kind of blacklisting whitelisting going on right now and we definitely wanted to contribute to this I think this could be a way blacklisting, whitelisting then because of our research many universities and institutions in Germany they do like FAQs and dos and don'ts so the community right now tries to help itself and I think this is a good solution but we will look in one year or one and a half year we will look again if the scientific community is able to cure this and I mean I'm not pessimistic probably because I think one of the we have to understand one of the reasons why this business model is successful is because it's also a business with shame you know you go there as a young researcher maybe or even as a established researcher you get your paper there you pay the publication fee you go to this conference and you find out oh damn this is some sort of a scam I fell for some weird concept of business practice but nobody knows so you go home they ask you oh how was the conference do you really say I was scammed sorry this was a big mistake I actually you know throw away government money or funding money many people don't have the guts to do this and they just say well it was okayish and try to answer in short sentences and hope that people talk about something else and so this is definitely a goal from our research the public knows about this and that there's awareness about this and I think after the publication it's hard to say we didn't know and also to encourage people to come forward even when you google best presentation award was it so many universities give out press releases you know having this like people are holding this I got a presentation award from was it because they probably were to ashamed to tell it home yeah please is there a model or characteristics of criteria that one could use to identify a fake publishing company so I see a paper that's cited you know published by this publishing company and well other than submitting a fake paper and having to have a garbage paper there's an answer to that and that is journals do have what they call impact factors and promotion and tenure committees do look at the impact factors when they look at the publications as mentioned earlier if they've got like 80-90% good publications they could slide some of these ones with lower impact factors into the university probably wouldn't even look at those papers to see how bad they were the question was if there are criteria so that I can see if it's a predatory journal or not so one answer is impact factors so of course you can look at the impact factors if they are very low or if they are not existent this might be a hint but there are also new publications or new journals out there for example so that there are courageous people founding a journal and making a good peer review but they don't have an impact factor so this is not the only criteria and I must admit it's very hard to definitely say this is a fake journal something I guess what we did for our research was we had more criteria so submitting a fake paper was only one but look at the website is there an office or a bureau is there an address, an official address and is it legit or is it for example some of them have addresses in London but if you google the address it's just somebody who sells addresses like virtual office addresses like this out there there is a guy who is called Jeffrey Beale he did a lot of research in this and he published something that is called Beale's List where he did his own research and published and made it public which papers and journals and conference organizes he deems predatory so again there is some problem with just a single person doing this and having a list as a definitive list but this is definitely a good hint he did some amazing job there they offer some black and white lists as well they are subscription based so for them it's a business model too for medical paper there is PubMed and there is also a white listing in India so what we did for the research was looking at different black lists from the country it's a hard job to find that out it's a hard job for scientists, for journalists but I think with these companies we researched I think you can be pretty sure for these companies if it's omics or was it or science domain for omics there you have the FTC investigating this for was it the CUNY case there is also an investigation going on in New York so this is also a hint for a predatory publisher please that's true it could run all day every day if you like there was a question over there first that guy had his hand up a while ago absolutely sure in London for example at the conference there was a guy he was really funny he was from India and he was pretty frank and honest and he was a professor so I think he didn't have anything to lose we asked him and he said of course I know this is a fake conference but anybody from India has to see London one time so of course there are people using the system of course there are and if you have 29 publications there you should probably be a little bit suspicious about the practice but the thing is we got some comments they were super superficial this paper was full of mistakes we just presented a few of them here we wrote it with the purpose of being discovered in a way and we did not so the comments we got back were so superficial that we thought this is absolutely no proper peer review but if you are like a scientist who considers the research great and the only review you get after two days which should raise some eyebrows I agree but when those only comments are this is an amazing paper you should just maybe change one sub headline somewhere you think oh I did a good job so many people don't have maybe the self criticism to see that this is not a proper peer review please yes very good question after how big it was and how many people are working there so we went to three VASAT conferences we had this international partner so I don't know how many VASAT conferences we were in total probably six or eight in South Korea and several other places and we think yes it's a family business and Borah is doing Europe we spotted him in Europe in Copenhagen in Berlin in London thinking his father he's doing Asia and also Turkey and then they have some kind of franchise in the US so this was not a family member we met in New York and we're thinking the daughter maybe is doing the books or something so she's doing the work behind so it's a very very small company and I think they can be a small company because there's not so much work to do so all that Borah was doing was just sitting outside the conference room collecting money and giving out name tags I left his laptop unlocked all the time by the way and you just upload your presentation on a USB stick so you just maybe I shouldn't have said that oh no it's no problem they had a funny story they worked with these 23 different media partners and I think there was one conference where there were a whole bunch of people who were buying bullshit papers at the same conference but none of them knew that the other so it was like a full conference of bullshit papers by these media they're like are you a journalist are you a journalist it was hard to tell please are we talking to who sorry the tax authorities have we talked to the tax authorities I don't know what the US tax authorities are but I did notify the FTC and the FBI and figured that at some point maybe they'd get around to something like that and I did that in a few countries as well not just the US but like Canada where they were having conferences last year and so on so folks are aware about it yeah there's a question there sorry I didn't was the question correct me if I'm wrong have we looked into how to deal with the avalanche of spam that academics are getting from these various publishers and how many people are responding and what sort of people are responding I've not looked have you guys looked at that it's hard to tell I mean we kind of think that most people who end up publishing there have at some degree responded to those spam emails that most academics know and we received in bulk after we made those fake and all these kind of things so we were also wondering who would reply to that but actually those spam emails they really vary in sophistication some of them are just hi you researcher you want published this great so okay but some of them were really like giving the correct titles citing some work that we have done saying oh we read that you did some work on cancer do you want to do this and this and this so some of them don't really look like normal spam and again if you don't suspect people to be that evil if you don't suspect predatory publishers to exist you could think this is a legitimate request for publication you think oh they discovered my greatness finally somebody found out what great researcher I'm doing and they invite me to publish oh yeah great I want to do this so some people definitely fall for this for this reason I mean I didn't get an email invite I googled it you know it's like political psychology it's like I'll look for a conference that looks kind of legit looked exactly like an IEEE conference so I wasn't on the wiser so yeah I guess people are searching and maybe finding the conferences that way too I mean we did eventually get all of the stuff removed once we started talking about it with journalists things went pretty quickly and then like a couple of weeks ago we uploaded the full paper to a prepub server osf.io and we'll be submitting the paper for the whole peer review process please yes we in the US this is the first yeah this you are the first who hear about this who was the lady you mentioned Rachel Maddow NSNBC no I don't know have we yes we teamed up with the New Yorker because the writer there he already did a story about this this fake predatory also yes yeah okay hmm yeah we will probably come off yeah write it down because we Germany far away from the US we work together though mother Jones picked up the research that we did and there is an institution an amazing one that you should support it's the International International Consortium for investigative journalism they so supplied us with some help connecting with other journalists and they gave us some technical infrastructure where we could share findings on an encrypted basis so that not everyone could read that the work that we were doing so and this is an American institution too so they work together with American researchers and they do a lot of research what can be done especially if you want to follow cases like the Malinkrod case or if you want to follow or the CO2 collision case because we did not research these cases until the end or the governmental cases and we wanted to leave you know something special for DEF CON so we were kind of we didn't want to have too much information in the US before DEF CON and so we were constantly worried about it and so now we're happy for it to be broad yes please also I spoke with our legal person from the station before DEF CON because this definitely is a Github project it's private but he did not want to cover for this so I would have loved to make this private project public here this would have been great but you know as these companies especially omics they're really willing to fight so I could not release it but I mean if there's a researcher who really is interested in doing this then just you know give me a shout out so maybe you can do something yeah it's pretty straight forward so if somebody happened to do that and put it out there oh that's awesome that would be a good idea alright we should come up with the name that's the next challenge so but the international journal of bullshit that would be my suggestion IJB BS correct yeah okay alright thanks so much for this this has been a pleasure