 Hello everybody here in the room and also I everybody who's joining us remotely It's my pleasure to introduce to you Kevin Chawinsky from Switzerland He's an astrophysicist from the Swiss Federal Polytechnical School in Zurich He holds a BA in physics and math from Cornell. He has a PhD in astrophysics from Oxford and he spent time at Yale Conducting research as a postdoc and later. I had to read up on that today as a NASA Einstein fellow, which is Just for the name pretty express impressive. I think Since 2012 he's been an assistant professor for astrophysics in Zurich at the ETH Where he conducts research on supermassive black holes In Switzerland his last name Chawinsky and maybe in all German speaking countries It stands for innovation and pushing boundaries And and here's why his dad Rosha Chawinsky is a media entrepreneur and he first came to like He was first known because he started a pirate radio station Broadcasting to Switzerland from a peak in the Italian mountains close to the border. So But here today Kevin is not to talk about his dad, but something else where he is really pushing boundaries He's going to talk to us about citizen science in which volunteers contributed their time for research They may for example Counts the birds they see in their garden or in other instances. They May help with naming something So citizen science something that's both conceptually and also in spirit and from its value is very close to us here at Wikimedia The title of his talk has everything to make Wikimedia and swoon, and I'm very excited about Kevin being here And so I give the floor to him. Please join me in welcoming Kevin Chawinsky to the Wikimedia Foundation. Thank you So, thank you very much for the great welcome Usually when I give this kind of talk about the the citizen science part of my research I have the Cognitive CERC plus slide from clay shirky about how how small the effort that goes into Wikipedia That goes into Wikipedia is compared to the total sort of idle capacity in humanity, but I figured you guys have seen this Far too many times. So I'm going to skip at this time I am going to open the with a little brain teaser since you are all expert researchers here at Wikipedia So this is the logo of my group, and it's because we were just called the eth black hole group But we have the same phrase eth black hole group here below in another language And if you can figure out what language that is I have a small price for you so to get started I have to give you a very brief introduction to Galaxy evolution so that I can then explain to you what the problem was that I was trying to solve When ultimately I founded the galaxy zoo So what you see here are the two basic types of galaxies in the top left you see a spiral galaxy and That's a spiral galaxy pretty similar to our own Milky Way galaxy. So it's a collection of about a hundred billion stars and Stars come in different masses in different colors and young stars shine in blue light and so you can see that this spiral galaxy is Lit up with blue light from young stars, which means star formation is ongoing in this galaxy Which means this galaxy is still growing On the other side in the bottom right there is an elliptical galaxy Elliptical galaxies are Fundamentally different beasts from spiral galaxies So you can see the entire galaxy seems to be made up of these red dim stars So these are old stars so that tells you that star formation in that galaxy has stopped and probably stopped a long time ago maybe several billion years ago and You can also see that the galaxy is no longer this sort of spiral disc shape It's now much more this amorphous football type shape so Somehow the universe transforms galaxies so star formation stops and shape or morphology changes and The big question in galaxy evolution is we want to know how this process works and You can think of two very general possibilities. You could either think You could first change your shape and Then turn off your star formation you could do that maybe by smashing two galaxies together and That would certainly change the shape of the resulting galaxy and that through some process Maybe to do with supermassive black holes could turn off the star formation and leaving you with this so-called red and dead elliptical Well, you could go the other way around You could first turn you could first turn off the star formation and then change the shape And it would look something like this. You would still have a disc, but there would not be any more young stars So when I was researching this question, I wanted To know whether in particular that the channel involving the mergers Whether that happens and whether we can study it in order to do that We needed to find the end products of these mergers We wanted to find galaxies that all already had this elliptical shape But they were still blue in light from recent or ongoing star formation So I wanted to find essentially a missing link in galaxy evolution and With basic calculations you could convince yourself that this would have to be a fairly brief phase Maybe just a few hundred million years and therefore there wouldn't be all that many of them out there Now the problem was that when Large galaxy surveys large samples of galaxies hundreds of thousands of galaxies became available people stopped Classifying the galaxies by eye and and they try to come up with clever computer algorithms or computer automated measures to classify galaxies and Most of these they either sucked Or they implicitly assumed that all ellipticals are red no spiders are blue So I wasn't able to find my missing link my blue elliptical galaxies So being an enterprising grad student I sat down and I took 50,000 galaxies and I classified them all by myself and This proved to be really valuable Scientifically I found my some of my blue ellipticals. I could do my thesis research But I also empirically determined the limit of how many galaxies a graduate student will classify it is 50,000 in a week So one evening in a key research facility called the Royal Oak pub just outside Oxford Astrophysics where I was doing my PhD I was talking to Chris Lintott fellow researcher and We discussed this problem. How are we going to go deal with large data sets large? astronomical surveys and We hit on the idea, you know, it was the the mid 2000s was 2007 And so it was obvious that we were just going to build a website and Put the galaxies there and then see if maybe some somebody wanted to help us Thus galaxy zoo was born We hacked it together. We got one of our web designer friends to be to design it We got an old friend of mine from from college Dan Andresco who's probably at the other end of the camera He built the back end We uploaded the galaxies and we wanted to see how it goes. So we had a very simple interface This is what it looked like. So by now this kind of looks really dated but in 2007 This was like at the edge of design It was a very simple design though because all it did was it showed you an image of a galaxy and Then it gave you a bunch of options a bunch of buttons to click to help us classify that galaxy by shape What is this galaxies morphology? We were not really sure How long it would Realistically take us to go classify the million or so galaxies that existed in this data set we hoped that Within five years or so each one of the million galaxies might be classified a few times and we would be able to do Research based on that. So that was our starting point. That's where we started out and we put the website live and we put out a press release and We launched and this is the list of most emailed stories from BBC news on 12th of July 2007 and we were number two scientists seek galaxy hunt help and You all know the internet. You cannot eat man flies to wedding a year early So the press release went viral and lots of people started clicking away on galaxy zoo And this is what our first few days looked like. So this the classifications per hour as a function of time and When we saw this We immediately knew that it hit upon something. There was some deep need That people had to go and contribute to scientific research The two features that told us that Are the following? Do you notice that right at the beginning right here? You see a few classifications and then you see nothing? the reason for that is because Not figuratively but literally the server melted We were serving the galaxy images off a server at Fermilab in the US And of course, we never anticipated what would happen. And so you're just serving them off of this server that was meant to be there for You know a few hundred astronomers around the world who occasionally will look something up So cable actually melted and the site was locked offline and we nearly died here And of course you guys are going well, you're stupid. You should have put that in the cloud So we have two excuses a we were stupid and B was 2007 So through a heroic efforts from some of our colleagues at Johns Hopkins University We put the we clone the server a couple of times put it back online And that's when the classifications took off So that's the first feature that's interesting about the spot The second feature is that within less than a day. We were doing per hour What I was doing in a week So this is really the the power of the crowd and this also shows you the the incredible demand the incredible Willingness that people have to go get engaged with science and again I feel slightly stupid telling this to this particular audience because Wikipedia is awesome and driven by essentially the same thing But we we were we were quite surprised So Once once once we proved that the system works and was collecting data We were able to do we were able to do two things With the classifications we were receiving So first thing we could do is we could tap into the so-called wisdom of the crowd so As is met with many things the consensus of a group of non experts is better than the opinion of one expert and so By the time we stopped each one of those million galaxies was classified on average by 70 people Which mean for each galaxy we had a consensus or a lack of consensus that was statistically meaningful and this allowed us to do lots of really cool investigations into galaxy evolution and Related topics and so just to dazzle you with some slides This is the kind of research that enabled it and I couldn't help myself So my my personal favorite paper that came out of this was Title the Green Valley is a red herring and this is a really hilarious pun if you're an astronomer so so getting the crowd engaged with your data is powerful and So the humans have a role in analyzing these large scientific data sets, but beyond All this collective intelligence that you can put to good use Processing large amounts of data. There's another thing that humans are uniquely suited to and they're uniquely suited to Spotting what a great philosopher once called the unknown unknowns So Scientific data the most interesting kind of scientific data are Things that you don't know are out there and so you're not looking for them And so in the olden days when scientists collected all their data themselves They would of course look at and and verify all the data. They were collecting but in the age of giant scientific experiments the Hubble Space Telescope the large Hadron collider the vast surveys of Ecology and in in various ecosystem ecosystems These things are full of potential discoveries that we don't recognize right now because all we do is run algorithms on them to Trawl these data sets That are programmed to find the things we already think of there So we were lucky. We made a design choice In the original galaxy suit that every galaxy once you classified it was linked to A web page that kind of looks like this that the survey prepared which contained all the ancillary expert data on this object And even more so it was then linked to all the other astronomical databases So once a citizen scientist said hey, I want to know more about this object What what other data are out there what other telescopes observed it what kind of observations were made it was all linked And this of course enabled The citizen scientists to perform their own discovery And the citizen scientists made some amazing discoveries They discovered things that we literally didn't know were there and only because they spotted them where we able to analyze them So, let me show you was a the poster child of the unknown unknown lurking out there in the universe and That's this object. So here's a galaxy Happen to know its telephone number. It's IC 2497, but it's completely irrelevant and This galaxy has been detected both in this particular astronomical survey and lots of others So depending on how you want to count this galaxy is being studied for between 10 and 30 years And so every algorithm that was sent was thrown at this galaxy said, yeah, that's a galaxy. It kind of looks like a spiral however When Dutch school teacher honey for uncle looked at this object. She said what the hell is that? There was a blue blob right next to it And so she went to Talk to the other citizen scientists. We had a forum. By the way, we created the friendliest internet forum of all time People apologize to each other and so so famously she she started the thread With the question. What's the blue stuff below anyone and then a link to to the object and before we we as the professional Astronomers ever got to it the citizen scientists started analyzing it and They named it after her they first named it honey's object, but then they asked her What's object or a thingy in in Dutch and she said for that So it became known as honey's for that before we ever got our hands on it and the the Really lovely part at the end this Honey's for what is not the only one there are other for vermin and so there's now a class of objects called objects So we got round we studied this object We looked at it with the Hubble Space Telescope So I think one of the coolest images my personally completely unbiased opinion One of the coolest images ever taken by the Hubble Space Telescope and and there you can see honey's for an all its glory So what what is this for verb? We now understand it much better It's a galaxy scale cloud of gas. So this green blob is about the size of the Milky Way But it consists entirely out of plasma and it's lit up. It looks green in this color scheme here I am not I'm not sure what it would really look like to human eyes probably just fuzzy purplish green. I don't know and this Galaxy scale gas cloud is lit up By the light emitted by a billion solar mass black hole at the center of that galaxy So this gas cloud is lit up by a quasar. So one of the brightest things in the universe The only problem is there's no quasar here, right? There's not a giant Galaxy brightness scale object at the center of this galaxy And this is because In the time the light from the quasar took to travel out to this cloud It's about 200,000 years light light is really not that fast The quasar shut off dropped in luminosity and 200,000 years for galaxies absolutely nothing. It's less than the blink of an eye. This is extremely fast and so honey's for that and the other for weapon Giving us a completely new way to try and study What black holes do how they affect their galaxies how they flicker on and off? It's it's incredible So we've already written a number of papers on this where we keep studying them and one of my PhD students is doing an entire PhD thesis on what we now call forapology So galaxy zoo was a success. It worked at collected data. It produced Clicks it engaged the community But we still had to prove something we had to prove that we can do Traditional science right in our metric obsessed world. We had to produce papers and citations and and we can do that We have a lot of publications many of them from galaxy zoo, but increasingly more and more publications from other projects, so What have we learned from this we've learned that citizen scientists can do two things they can analyze massive data sets and They can also perform independent discovery. They can go make their own group their own collaborative They can get together and study a subject and do their own science and This means that citizen scientists make your data more valuable It's a certainly true in science and it's also true in in any other field You make your data publicly available and give people the tools to grapple with it to analyze it They will do interesting things with it and that makes the data more valuable. It's in a world where governments and taxpayers pay for Extremely expensive scientific experiments that hopefully will transform our lives and our understanding of the cosmos You know, there's a very inexpensive way to make that data More valuable by allowing people to go analyze it by themselves We also came up with some ground rules for doing citizen science along the way and I'd be curious to see if you guys To what degree you guys feel this lines up with with how you engage with the Wikipedia community So the single most important rule for me is that you have to tell the citizen scientists what the research is about You can't just say here's some data. Please. Please click away at the following and then get lost You have to tell them why the research is important what it is about and why they're doing what they're doing It follows I think quite naturally from this is that you treat citizen scientists as your Collaborators, they're your colleagues. They're not your minions. They're your equals and you they deserve your respect and they deserve your honesty And again straight from that follows that you do not waste the citizen scientists time So you should not just launch a citizen science project and tell people to click away if you're not sure That what you're asking them to do is actually valuable So if you're not collecting data, so you've got enough clicks you should say, you know, we're done taking clicks You can keep using the site, but we're not recording this anymore Similarly when people started coming to us and said hey, we want to we want to build our own type of galaxy zoo like project Can you help us? one of the first questions that that we ask is Okay, so you want to build a citizen science project What's the title of the one paper that you'll be guaranteed to be able to write if you run this project? and the reason we ask that question is to make sure that This potential project does not waste the citizen scientists time So there are benefits to having citizens involved in scientific research The first benefit is that it of course makes people more engaged with science Right, we all talk about how important is to be a scientifically literate Society but before you can have that you need to get people interested in science. It actually matters and And it can be hard to predict. Well, you know, who's gonna be interested engaged by you know Astronomy and who's gonna be engaged far more by something more real world like the quality of the water or the population of birds in in their neighborhood and Of course having your citizens involved in citizen science makes them more scientifically literate And what I mean by that is not just in the sense of they learn facts But more in the sense of that you turn them in you help them Think like scientists you teach them the scientific method you teach them to think Scientifically, how do we know that an effect is statistically significant? How do you design an experiment that is meaningful and? by having the citizen scientists directly involved in this process as collaborators You're teaching them to develop what I call but so when I do this with my students in university I try to do the same thing. I call it giving them a BS detector So when a politician or a business or whatever comes to them and says here's an outrageous claim They are able to evaluate that claim the way a scientist would know. Where's the evidence? How statistically sound is it? Where are the biases? so citizen science is a great way To teach this way of thinking to people so we've learned from galaxy zoo and We built this organization called the zooniverse and this grew essentially out of this Phenomenon where other scientists and not necessarily astronomers came to us and said hey we want a similar thing can you help us build one and We did we we got a bunch of grants and hired a bunch of people and We built the universe and so we are out right now around 50 live projects and I think this is actually outdated Since I made the slide it. I think we're now by at about 1.5 million citizen scientists taking part in various projects and There's There's a warning There's a bunch of things of course we learned about citizen science Okay There's a bunch of things we learned about the citizen scientists, which is We originally thought that most people will be interested in most projects and sort of hop back and forth between them But that turns out not to be the case it turns out that most Citizen scientists they have their one project that they love or maybe a small number of projects and they'll stick with it The distribution of citizen scientists also follows kind of the the purito principle So a lot of sign citizen scientists do very little and then a few citizen scientists do a lot And I bet that's also reflected in people who edit Wikipedia There's a few people who edit a lot and a lot of people who contributed a little bit This is this is also true of the citizen scientists The next step in the evolution of the universe and citizen science is system called Pinocchies which was funded by a Google grant which Which essentially a Web-based platform where you can go and build a citizen science project So in in about half an hour you can have a basic citizen science project up and running Probably take you a whole afternoon, you know to get all the content and design and upload all your data to get all that Right that maybe takes you an afternoon, but then you have a citizen science project up and running and anybody can do this You don't need anyone's permission. You can go build it right now There are also ways where you can have The Zooniverse Help evaluate it and promote it say put it on the front page of Zooniverse So there's some vetting process that goes on there But right now you can make one of these projects and you can even say I want to keep it private I just want to share it say as an internal tool within an organization like the Wikimedia Foundation Or I just want to mail it to my local neighborhood, or you can just make it completely public and let anyone who wants to contribute to it It's completely up to you So this is this is where we are with citizen science. I think we've We've solved the problem of I have a bunch of data mostly images But we have some non-image data projects and I need some collective intelligence I need some humans to go help me sort and analyze those data That's the solve problem. We know how to do it. We know how to do it very well It works so well that with with this project build a tool You yourself without knowing very much about what goes in in the back end can build one of these very very quickly So so where do we go next with us? what's the next step in the evolution of citizen science and So what I'm giving you now is my personal vision of how it might pan out or how part of the Evolution of citizen science what that might look like so we have a problem and This is a more general problem and I've stolen this argument from a colleague at ETH who's a professor actually of computational social science His name is dick Helbing and he pointed out the following So we all know because of Moore's law processing capacity doubles every 18 months But data volume doubles every 12 months And so a lot of people say ah, we have a we have a big problem because we have to diverging exponential state of volume grows faster Then our ability to process it grows And what dick Helbing pointed out to me was that that's actually not the real problem The real problem is much worse than that And that's that the data complexity goes up factorially doesn't go up exponentially And if you remember your your your mathematics Factorials go up even crazier than exponentials so we have this extremely rapid divergence between data volume and data complexity And on the one hand and our ability to process data on the other hand so I think citizen science in collaboration with machine learning Can help bridge that gap somewhat it's going to be one of the tools in in our tool belt to try and tackle this problem I'm sure lots of you guys are thinking about similar approaches about how to get machine learning involved in in these data challenges So here's here's where I am trying to go next With the help of citizen science So this is what we're doing right now. We are using Google's TensorFlow deep learning Platform with the sci-fi algorithm, which by now we've modified beyond recognition. So it's a convolutional deep neural net and we're using it to try and reproduce What the citizen scientists did in the galaxy zoo? so we feed these feed this neural net we feed it images of galaxies and The classifications to train it and then have it go classify yet more galaxies Because right now we're at a million galaxies But pretty soon we'll be at a billion and maybe a hundred billion and then there's not enough people on the internet to go Look at all of them. And yes, we translate galaxy zoo into pretty much every language available where we have a volunteer So we train the steep neural net to classify galaxy images and our our twist on it is we in the training set we don't train on Absolute galaxy classifications, but we train on the vote distributions from the citizen scientists But another way we are training a deep neural nets to go behave like crowds of people And so the performance of this is pretty amazing. So it's just a snapshot of how in about 24 hours on a high-end Mac Pro we the Accuracy of the precision of the the neural net reaches a pretty amazing level and what I mean by pretty amazing is the following We ran a Kaggle competition to go mimic the galaxy zoo citizen science results two years ago and Without essentially much modification without too much Customization we are already beating the winner of that Kaggle competition who by the way Santa deal him and he now works on deep learning at Google So we're building this capacity right now. We're learning on the go And we're about to deploy this technology to the National Supercomputer Center. So that's going to be really cool So What I've told you about now is How we can use the citizen scientists to train the machines to do what the citizen scientists did before So the citizen science scientists are still necessary to train the machines but the machines are necessary because of the data volume But all I've talked about is How do we go about? classifying Things like galaxies to get the same kinds of vote distributions that I talked about There's of course another thing that the citizen scientists are great at which is anomaly hunting What are the unknown unknowns out there right? That's just as cool in science and So another thing that we're thinking about is how do we how do we get? The machine learning to facilitate this anomaly hunting. So if we in the future have No more no more no longer million galaxies, but a million billion galaxies How do we how do we go find the the weird things that in the universe that we don't know about? If there's too many to ever show them to the humans So we're hoping that we'll be able to filter down These large data sets to something more manageable say the 1% or so of galaxies that are anomaly candidates and Then we can take those Feed them to the citizen scientists Who can evaluate them and really tell us whether they are? genuine anomalies or whether they're just a slightly misclassified galaxy in which case We feed that information back to the neural net and improve it Which leads me to the final step? How are we going to deal with enormous data sets? in science I Think about it for science, but of course this kind of system would be applicable to lots of other things So imagine you have a data source like one of these large future telescopes or whatever it be You put that data on a server. How are you going to analyze it? So here's Here's my my vision or my Attempt at trying to build a system It lets us deal with this kind of data so There's going to be a dispatch algorithm which takes the data as it comes in You know, we just received another million galaxies what do we do with them? Okay, so we send some of them to the humans They classify them and we use those classifications send it to say a deep learning module Build a training set and trade and the next few million Just all go to the machine learning Which then verifies whether any of the objects coming in would help it improve the training set if the humans Classified it for them. So we send it to the humans and they classify and they return the answer to the neural net You can also now do this anomaly hunting essentially on the fly because of the deep neural that the neural net thinks It's found something that's a candidate anomaly it immediately pushes it to the citizen scientists to be evaluated and pushes it back You can go even a step further now imagine instead of a static system This is a flowing system the data keeps coming in you could then link this the following way where say The either the deep neural net or the citizen scientists decide look We would really benefit if you observed or re-observed that kind of object Do you send that request back to the data source to the telescope or whatever it be and it can then perform that observation to help improve This virtuous feedback loop between the humans and the machines So this is why for a recent funding proposal I I came up with the term for this of citizen science cyborgs because both the humans and the machines are absolutely necessary to the success of the system and It taps into the things that both the machines and the humans do well And of course, there's no reason why you would stop at having one citizen science project and one Machine learning module plucked into this kind of system. You could have many and I'm going to stop here And maybe you have some questions. Thank you Thank you for that. It was really fascinating and a question what you you talked about using citizen scientists sort of to Help to help the machine learning thing Detect anomalies like the machine would would say if you say like say these are the 1% or so that are anomaly candidates And then show those to to citizen scientists who would then Make their own determinations. Do you think the citizen scientists would? Do as well detecting anomalies if they're only looking at potentially anomalous Data as opposed to they're looking at all the data and then so the anomalies might stick out more so of course they wouldn't do as well because By definition if you did if you don't know it's an anomaly you can't forward it to be checked This system is only Relevant and useful if the data set you're looking at a so large that you can't crowdsource all of it So I have a million galaxies. I know I can handle that with citizen science alone But if I have a billion there there are literally not enough Internet users on in the world that I could go and try and recruit to help me sort through that and so of course this anomaly hunting system I describe is Inferior to having humans look at everything. I Should add to that when it's talking about honey sforva and honey for knuckle. She wasn't even the first human Citizen scientists to look at that image. She was the 26th. So it wouldn't even help you to have each Object each galaxy be shown once to a human that's clearly not sufficient Now you're talking about if you have a billion galaxies and you need to have each of them you'd on average 26 times before the anomaly is noticed You know it the problem compounds. So I have a question from the I have a question from the IRC which is Some curiosity why some of this knowledge doesn't necessarily flow back out to the public and in particular it seems like very few Astronomical researchers edit Wikipedia and help work on those articles. They're curious. What do you think about that? So there's a question why why do not more scientists astronomers? Whatever. Why don't they spend more time editing Wikipedia? I Think you know, it's it's always tricky to speak for a group and generalize I think it's because ultimately in in the academic world. There's Zero incentive zero credit for doing it If I could if I could spend half an hour editing Wikipedia and contribute to humanity that way or write another grant or another paper I know what I'm incentivized to do That that doesn't mean I know many astronomers who do edit Wikipedia But it's probably not at the level that you would want other questions. Otherwise. I have one So this system it's pretty complex right with with tensorflow adding a lot to that and Algorithms getting complex or as they learn. How do you make sure that citizen science is still feel like? Important in all this system like when you when your goal is to keep the the population also like in the loop to Make of the population scientifically literate. How do you keep them in there? So I think that that's a big challenge I mean this such a system would rely Critically on the humans that are incidental there a critical part of the system And so you you just have to you have to find clever ways To communicate that to make it clear that even though the humans may look at only 1% of the data That them looking at 1% of the data is what enables us to to understand the rest and so yeah There's a big challenge ahead for us to to communicate that and communicate it well Especially now that more and more people are worried about you know machines making their their jobs obsolete. I think that's going to be a struggle As you deal with more and more different citizen science projects and proposals through this universe and things like that What do you? I guess there are a couple questions one does that Platform enable and or allow for any sort of incentive structures like you get points for rating things or whatever And and what is what is your attitude towards those sorts of things in general what they like benefits drawbacks trade-offs So gamification is hot and contentious topic in citizen science There are people who not just champion In gamification, but the the project relies critically on it. So there's projects like fold it has anybody here done fold it So this is where you you fold proteins and you're trying to optimally fold proteins So so that it does what it should do become a new drug or whatever and and so It's a it's a competitive enterprise And so the the players the citizen scientists they compete against each other to you to have the optimal protein folding and so there The competitive element is I think absolutely central When you're looking at things like the zooniverse style projects where we are relying on the humans Doing an analysis and doing that analysis. Well, I think it very quickly becomes difficult In the early days of Galaxy Zoo, we we try to introduce something like this and we put up a leaderboard So you put up the list of the top 10 classifiers just in terms of who classified the most galaxies and All 10 of those emailed us and said they were convinced. They said the other nine must be cheating and so I think that that in this case added an unhealthy element of competition to citizen science because Then you're sending the message that would the users should do is classic click as quickly as they can Which is not what you want scientifically, right? You want people to spend their time and think about, you know What really is my opinion my judgment on this object? And you also don't want Casual users who maybe show up just once or a few times or occasionally do a little bit of citizen science You don't want them to feel like their contribution doesn't matter Because it does matter because when you look at Of course, this depends on which Project you're looking at but certainly in galaxies you the bulk of classifications does not come from the super users It comes from the casual users So you don't want to make them feel like their contribution doesn't matter and so In general as a rule of thumb, I I don't like gamification But it clearly has its applications So you have the task like folded that is intrinsically competitive then then it makes sense Have you seen projects with gamification also have like People who have done it more start to claim like expertise as if they're like oh, I've I've rated a thousand galaxies Therefore my rating is more valid than you're one so We looked at our our people quote-unquote better over time and of course better in a scientific past right in the classification task What does it even mean because we don't have the underlying truth? So you could you could do the following you could work out How long does it people take before they generally agree with the crowd in general and it turns out that? most people Approach more or less the same level of expertise very very quickly And and then they plateau and I know they plateau because one of the most prolific galaxies you classifiers he's saw he's seen every single one of the million galaxies and He's terrible at least in the sense that he disagrees most of the time with everyone else So looking at a million galaxies did not improve his galaxy classification ability so I I'm not I'm not a neuroscientist, but I would bet that My explanation for this is it's a pattern recognition task Human brains are fantastic at pattern recognition. It's an intrinsic ability. We have and so You have a new type of thing to classify we we build up that capacity very quickly and then you know, it doesn't need to improve any further so There doesn't seem to be such a wide dispersion In tasks that deal with image classification. I actually have another one So as you as your as your base of citizen scientists grows And you said you have the nicest forum there is on the internet I'm wondering does it stay as nice does it is as as academic does academic rigor that sometimes can also be a pre well rigorous And does that translate into your forum due to people come in and this is close to do other questions like What's the motors operating there? so it's The MO in in our forums in general is very supportive. So whenever we Whenever there's a new project, we make sure we explain to the scientists that run that project look during the launch during the first week Or so you have to be very very present in those forums because you have to see the community knowledge, right? If there's a particular kind of Thingamajig in your data that actually is an artifact and not an alien so example in galaxies you there these weird green And red and yellow trails across the sky and people say oh look I found a UFO and And then the astronomer comes on and says actually what you found is a satellite or an asteroid It's actually kind of cool Just not an alien and so so you have to impart that knowledge And then once you've given that knowledge to sort of the core Community members they will then transmit it further into the community And I think the same way we started off trying to to to have a very positive and inclusive online community and that's just stuck with it and that We established a nice and light tone right from the start and people stuck with it and they liked it and so Yeah, I don't think there are many places on the internet where people apologize to each other But the galaxies who form is one of them any other questions, so you said that you want it to be possible to Classify a billion of galaxies But how are you going to search for them? Is this something that similar techniques are also applicable? so Yes, I mean we're trying to build this kind of system in anticipation of Telescopes the facilities Coming online because they are they're planned many many years ahead of time So for example, the US is leading a facility called the large synoptic sky telescope or LST so this is going to be a 6.5 meter telescope based in Chile and it has an enormously wide field of view and it will image the entire southern sky every three days and the data volume that this telescope will generate the scientific volume that the telescope will generate is Is actually changing the way astronomers think about the science because the core team of people working on LST They're they're not really astronomers anymore that data scientists and computer scientists and So this this is sort of this is our approach to say look here's one tool in your tool belt that you're going to use going to be able to use to analyze these enormous data sets when they come and The humans can and will have a role in this Yeah, thanks for the talk of this was really fascinating and I think I really appreciate how Well thought through your system is around the work that you do I think Wikipedia is as has a lot of scale But is sometimes missing the like systematic analysis of how it functions and I mean there's a lot of mysteries inside of Wikipedia And it's very enlightening to sort of compare it to your project. I Wonder one thing that kind of strikes me is like galaxies are cool astronomy is Interesting, I think it's sort of easy to get motivated to like look at the stars and think it's a kind of like sci-fi, right? I wonder if you've looked at any kind of more boring citizen science projects and If you see similar motivations or different motivations or maybe the cool factor has nothing to do with with Why galaxies you seem so successful? So so this is an excellent question and we thought a lot about this because we started with galaxies because yeah, they are cool By now we have a whole range of citizen science projects in many different disciplines and They all find their audience and I think the the reason for that is because it doesn't matter what the subject is There are people out there who care about it And that's why the Wikipedia page of you know, ancient Sumerian kings of content because somebody cares similarly when you when you do a citizen science project on ships logs books that contain climate records from a hundred years ago I Personally, I don't find that fascinating. It's very useful because it's vital for climate science But there's a community of people out there That's absolutely fascinated by these ancient log books and and and that's good not not everybody has to love every citizen science project You just have to find one that you like and the internet's a big place So the space for almost any any interest Yeah, are there any other questions on our RC Jacob? Okay. Well that in in the spirit of agreeing that there are many niches on the internet and that everybody can love their thing on the internet and contribute to free knowledge and science Thank you so much for coming. This was amazing And I'm sure we'll be in touch and talk about this some more and also maybe you can do an edit on Wikipedia about this You know, all right. Thank you so much Kevin