 No, yeah, I said yes to do this plenary and I appreciate the invitation I'm very excited to do this. This society is very exciting for me, I feel very at home and evolutionary ecology it's it's one of my identities the other being an anthropologist but I'm 100% anthropologist and 100% an evolutionary scientist and so I'm very happy to speak to all of you on the topic of how we can make incremental or dramatic improvements to scholarship and particularly an evolution in ecology. And I know, probably a few of you are here because you like chickens, chickens are great. And I promise to talk about chickens, but first I'm going to talk about hockey. So, hockey is a peculiar North American sport. I assume most of you know what it is but let me do a very rapid description. You take very large aggressive men put them on ice skates and give them a very hard object to hit with very long hard sticks. And then they try to hit one another with both the puck it's called which is a hard piece of tire rubber essentially, and the sticks. There's a lot of minimal safety equipment and there are lots of injuries in this sport, and a lot of the fan base likes the aggressiveness of this sport. It's gotten more domesticated over the years but in the, there was a bunch of safety reforms in the second half to 20th century. During that period. There were no helmets worn in the National Hockey League. But here is a few stills from one of the most famous fights that happened on live television in 1969 between Ted Green and Wayne Maki, where a stick fight broke out and this happened all the time by the way in hockey at the time. And in this particular case, Maki eventually hit green on his head with the stick and shattered his skull on live TV it was quite dramatic so dramatic that the tape was deleted. And all that survives are these stills which I hope are not too gruesome I selected them so that they would not be. This was one of the events which catalyzed the safety movement in the National Hockey League, and about a decade later helmets became mandatory, but lots of really interesting debates about reform happened in the in that decade. And here's, there's a nice short history of it in Shelling's book which is published in 1978 right before the helmet helmets became mandatory. This is a fantastic book called micro modus and macro behavior which is all about human institutions and how they evolve and guide human behavior. So this is just a nice quote I hope you'll indulge if I just read it for you. One player summed up the feelings of many, it's foolish not to wear a helmet, but I don't, because the other guys don't. Really, but most of the players feel the same way. If the league made us do it though, we all wear them, and nobody would mind. Here's an image of two quite famous players from Chicago Bobby holds Dan Mitko showing the classic hockey player dental arcade. So, why am I bringing up this example you probably guess that there's a metaphor looking here about academia. There's a general point about incentives that I'm going to draw out later with some examples. But for now let me just say, often there are conflicts between levels when we talk about incentives and the incentives operating at the level the individual stop individuals from actually doing what's good for them. And here we have a case which is quite common in human institutions where the individuals recognize that and they asked for some top down solution, because they know they can't themselves fix the system. And you can probably guess why I'm mentioning that. Okay, another quick example, I'm going to run through some examples here playfully but I'm going to draw out. I think some, some serious points from the collection. Second example is about Protestant churches in the United States. So the Americans in the audience know for sure. And most of you who are not Americans probably also know that the United States is an unusually religious Western democracy. And it has a very dynamic and even entertaining to put a positive spin on it. A set of religious dynamics with competing churches and it's very dynamic actually, compared to say, Western and northern Europe where religion is if you'll forgive me quite boring. There's a state church, you're a member or you're not. People don't talk about it. Right. And, but the United States religions a big deal. And it's incredibly dynamic and one of the biggest changes in recent history in American religion has been the decline of the so called mainline Protestant churches. Mainline Protestant churches would be the classical ones Lutheran Methodist Presbyterian as a Scottish American I grew up in the Presbyterian church. These churches have declined quite rapidly and is beginning in the second half of the 20th century and it's still continuing Methodist congregations Presbyterian congregations are just emptying out. And in at the same time evangelical churches Baptists Pentecostals and the like, have increased certainly held their own as a in this graph as a proportion of the total, but they've spread as the population increases. And you might ask, what accounts for this different in the difference in the fates of these two religious traditions. And one idea would be, well, the mainline churches are just losing people so called apostasy rates are higher. Or they're worse at recruiting than the evangelical churches. And it turns out neither is true. The evangelical churches and the mainline churches are just about equally good at both losing and gaining members of the congregation. Neither is is better than the other, for the most part, there are small differences in some regions but it's mainly the same. And the big difference as documented in this great book called the Churching of America by thinking stark is birth rates. The evangelical churches have grown as a proportion of American Protestant churches because they have bigger families members of the congregations have bigger families and pro natalist norms are instructed at the pulpit. Preachers talk about the importance of having kids and growing the church, whereas the mainline churches tend to emphasize education and upward mobility into the middle class, which as everybody here knows, education tends to work against starting a family early, and you end up with smaller families as a result. And this explains the vast majority of the difference in fates of these churches. And this is, it's not about incentive to join the church or not to leave. It's just how many kids you have, and then the kids are socializing the church, some of them leave the church, some of them join other churches, but even a fairly modest difference in birth rates over decades can lead to mainly the replacement of mainline churches and many parts of the United States. So, yeah, I'll draw out this lesson more later for now just bookmark it in your mind as incentives aren't the story here but they've been a major cultural change in American religion. So, let me step back a moment from my playful examples and say, be transparent as it were about what my goals are. And I think scholarship is a big tent, and we should allow lots of diversity I mean essentially scholarship is kind of an anarchist society. But anarchy is also subscribed to common principles and goals and I just want to state mine. And if yours are slightly different that's fine I imagine we mainly overlap. I call these my four pillars of research you can picture them there on the right that's research being held up by its four pillars. The first of these is that we'd like, we'd like scholarship to be reproducible that means we could look at an individual paper or study and actually verify that the result exists it's not a product of an error, or fraud. So, let's say same investigation repeatable inference, the reproducibility standard which is extremely important. We'd also like it to be reliable. So for a new investigation we'd like to get the expected results so that this could be generalization to the population that the sample originally came from or it could be transport to some new target population. But, but if we understand causal effects generalization and transport or should be possible. The research should be reasoned by which I mean we can justify the inference so that is one of the worst things about doing science as opposed to just living your life as an animal and in a natural environment is that it's not enough just to be right about something in science you also have to be able to justify that to your peers. And so knowledge is justified true belief it's not just true belief and so the reasoning part of it showing that results follow logically, and you didn't just guess something. I think it was true empirically that's part of the scientific endeavor that's one of the most challenging things about it. And then finally, but certainly not least important. We want our culture to be respectable, by which I mean, ethical and open. We should do no harm to our subjects and participants, and we should preserve the rights of our colleagues in the public to reach their own conclusions about the meaning of our research. And we have an obligation to create humane I use that term quite generally working conditions for scientists themselves at all stages of our careers. So incentives have been a real focus and thinking about reforming academia and scholarship to meet those types of goals that were on the previous slide. I've talked about this myself and I work at this in my own department and in the mox pond society actually as well. And a lot of this means that we emphasize and educate researchers on on the benefits of adopting open transparent workflows, and there are real significant individual level benefits to doing the right thing right it's like it's it's this basic dilemma in life and all the parents in the audience will understand this because you tell your kids about this all the time, you can pay a short term cost and get a much bigger long term benefit. And a lot of science is like that use version control now, future you will thank you for the effort. Yes, it's a pain right now, but future you will be better off. And educating the research community on those individual incentives is an important thing and that's that's emphasizing positive incentives, and then we want to reform negative incentives things like the use of h index in in in promotion and hiring decisions and so on, or replace them with other kinds of incentives for transparency. I think all of this is important and I support it completely. But it's a very small part of the general dynamics of how the scholarly community works and how scientific beliefs evolve. And so I want to spend the rest of the time that I have, providing some examples of these other effects and stimulating your conversation about them so that we can bring this kind of broader system level analysis to the reform movement, because I, there are parts of human society to do this like policy impact analysis and essentially what we're doing is we're we're trying to do niche construction of how scholarship works and some policy systematic policy analysis would be good. Okay. And one example here that I couldn't give a talk like this without mentioning is is the basic fact that incentives are only one motive for behavior for humans and there's a big social science literature on this so there's now I'm putting on my anthropologist hat if you'll indulge me for a moment I'll talk about chickens chickens are coming. And for now let's talk about children. And imagine you ran a daycare, and the parents are leaving their kids with you while they go off to work during the day, and then they, they pay you money for this you educate their kids potty train them all kinds of things. And then the, at the end of the day the parents come and pick up their kids and take them home. The chronic problem in this sort of business is that parents are late to pick up their kids it's not that they don't love their kids and want them it's just that you know they're late things happen. They're competing incentives, and they know their kids are safe with you so maybe that they're not in a hurry to save their kids from you, which is a good sign. Nevertheless, this is very costly for daycare centers and, and there are whole businesses that have sprung up just to help daycare centers deal with tardy parents. There's also a scholarly literature on this and that's where I want to bring it in it's relevant for us thinking about scientific reform, sometimes appealing to incentives is counterproductive. And so let me walk you very quickly, it's just one slide here on the study walk you very quickly through this experiment which is now a classic in the social sciences in in thinking about institutional design. There's a paper from 2000 called a fine is a price. And what they did is they did a randomized experiment in Israeli daycares, where they selected a random group of daycare groups to have a fine so every minute that a parent was late picking up their kid they would be fine to a certain amount of money. Previous to this all day care said not had fines there was just a norm. Here's when you pick up your kid and if you're responsible parent that's when you pick them up and, and so on and then when parents would still be late but they'd apologize. So in the first five weeks what you see on this graph on the horizontal axis is weak number in the study is a 20 week study. And average number of later rivals. This is number of parents on the vertical in the two groups randomized groups. The dark trend being the groups where the fine was instituted and then the control group is the right squares. So the first five weeks it kind of looks like the fine is working it's not having a big effect. But there's slightly fewer late parents in the first five weeks, but then after the fifth week everything goes wrong. And what happens here, and the researchers interviewed the parents about this it's a very rich study is once you make it a fine well that's an explicit social contract and if the parents are willing to pay the money. Well then they can leave their kids there as long as they're willing to pay. And so you get more late parents actually and more inconvenience for the daycare center by instituting the fine, because it was no longer an ethical issue you see. So why am I mentioning this well it's a great study and everybody should know about it. Effects like this have been documented in lots of contexts now not just in daycares, although it's not always found you have to have the ethical norm in the beginning. If there's no ethical norm against being late in the beginning then the fine isn't going to be counterproductive right so you have to have some ethical norm to begin with. But I mentioned it because it's a classic but also because they're they're context within the sciences within all human institutions that are like this where it's okay to appeal to people's ethics. Sometimes it's just true that people have to do something that's costly to them individually because it benefits the system, and it's not wrong to appeal to that I think that's just my opinion. Right. So, let's come back to hockey and churches now and I'm going to tie these into science reform and yeah chickens are coming, they're coming, those of you the chicken fans in the audience. I think evolution ecology actually has a lot to contribute to broader discussion about the reform of academia and that's because evolution ecology is a field that grapples with systems and the mix of forces that create system dynamics over time. This is our bread and butter. This is what we do. And it's not enough to understand individuals or rather the insights and evolution ecology come from understanding that the properties of individuals are caused by the population and vice versa. Right, there's this whole system perspective that's one of the things that really attracts me to the field. So, I'm going to use the hockey and the Protestant churches examples is just memory placeholders for you to talk about two broad categories of forces that are related to incentives but are not part of the traditional discussion of incentives but are arguably big deals in the science in science. The first is structural incentives that is how the structure of the society creates incentive like forces on human behavior, even if individuals aren't aware of those incentives so we wouldn't traditionally call them incentives because the usual definition of the word incentive is it something that's consciously there's conscious awareness and deliberate adjustment, but nevertheless it has a big effect on individuals. And then the second is is population dynamics broadly so demography and development are forces that can compete with individual individuals adjusting their behavior to meet incentives and can even overwhelm those incentives. So let me run through now chickens are coming. Three playful examples, which I think have nevertheless serious ideas embedded in them. And the first is to talk about chickens as an example of structure the impact of structure that I use a feral sheep. And then the third force is more broadly, and this will slide quite rapidly into talking about kitty wakes and the impact of luck in in populations where there are really strong survival bottlenecks like kid wakes and academics. I've previously done some work on this with with Paul small dino. This paper we published. Now I forget when 2016 or so called the natural selection of bad science and I'm going to spend a lot of time discussing this because I've already published it just to put it here in case you haven't read it. If you are interested in reading it and want to ask me some questions about it later I'm very interested in your views. The basic idea of this paper is to highlight what I think is a general phenomenon in human institutions but especially in academia is that a lot of the bad behavior in academia is not deliberately bad individuals aren't cheating consciously, but nevertheless they have behavior which undermines the scientific project, and the forces that have created that behavior are due to the incentives but the incentives aren't consciously held their structural. So here's the 32nd version of this argument. The questionable research practices that's what qrp means are in fact normative in scientific communities people were taught them by their peers and their advisors to reinforce by journals. And what these things tend to do is create results right often false results but they create results. And, you know, p hacking and outlier dropping and hypothesizing after results are known and all manner of other things. People were instructed to do these things and you can find articles in which prestigious professors advise their junior colleagues to do these things. They're not hiding it right that they'd say because they think it discovers the truth. This is what they believe. Because there's a, they're not aware that the incentives are selecting for these questionable practices, and then peer review and promotion review and such leads to more success for individuals and teams that practice these things. And that has made them normative despite the fact that statisticians for example have been complaining for almost 100 years now about p hacking. So every decade the American statistical association publishes another please don't use p values like this paper, and it just has no effect. Because the, the, the selection going on in the sciences is very powerful I think anyway this is this is the structure of the argument. It's not that this is always true but I think it's an important dynamic. Now I'm going to talk about chickens, and here's another important dynamic that is also a consequence of structural incentives. And what I mean by that is the level at which awards are applied. So, probably not many people here have kept chickens. When I was in Davis there were, we were allowed to have chickens in town and so lots of lots of people had had chickens and running around their houses laying eggs for them and chickens are great. Actually, they're very affectionate animals. But not all chickens are affectionate some of them are cannibals. And this has been persistent problem in in foul domestication is aggressive behavior between chickens and cannibalism they eat one another's eggs they eat one another's feathers. Sometimes in in strong cases they'll peck one another to death. And so, one of the ways that this was traditionally managed and is still managed in some parts of the world is by beak trimming. So on the left here you see what a chicken's beak should look like it's pointed, and, and then on the right you see a chicken gets had its beak trimmed and so this makes it very difficult for it to peck aggressively other individuals or eat feathers off off the other pins. And so this is a way of managing the aggressive behavior. But starting in the 80s. In the Midwest of the US I think this revolution began of changing house hens are selected to breed out the aggressiveness so that they didn't have to trim their peaks trimming peaks is bad for the birds. It makes it hard for them to eat, and it can cause lots of secondary infections they can really can't close their mouths. Sometimes they have trouble drinking. So, here's a classic paper from 1996 on the group selection for adaptation and multiple hand cages. So the historical background is it's very economically expensive to house hens individually and so they're they're housing groups and but traditional animal breeding is you'll you'll select the most productive hens. And those are the ones that produce the most eggs, and those are the ones that you selectively breed for the next generation. And what this does is it favors cannibal hens. Because the, if you're the first cannibal you're more productive than your colleagues let's call them in the henhouse in the chicken coop. So the recommendation here is to select the whole coop as a whole and instead favor family lines which have the highest average productivity, and this dis favors cannibalism so again here, the the effect sizes here I've highlighted in the parts that I haven't blurred out are really extraordinary you go from mortality going from 68% to 8.8% from 169 to 348 eggs per hen per day from 52% 68% eggs per hen house from 91 to 237 and egg mass from 5.1 to 13.4 kilograms. There's no trade off right it's it's this incredible effect where the whole system is way more productive but you have to apply the incentive that the group level so that you get traits which are group cohabitating compatible. Of course, this is evolution biologists understand group selection right we, we, we saw study it in our intro classes and we know about multi level selection now and there's this very productive formal framework for understanding how it works and when you do artificial selection of course you you basically play in God and you can control which parts of the multi level decomposition are most important to selection and so that's what happens in the chicken coop as you select the group level is at the individual level. So just the cartoon version very quickly, the change in, this is the so called price equation, the multi level price equation, the change in any trade P, in this case cannibalism and integration can be decomposed to the within and between group effects the between your perfect is exactly the covariance between the trade your cannibalism and group productivity group fitness. That is the average fitness of members of the group. And the within group effect is the average covariance between cannibalism and fitness within groups. The important thing to realize is for an anti social trait like cannibalism. It is disfavored by the between group component because cannibal groups do worse than non cannibal groups. Yeah. They survived less they lay fewer eggs, etc. But if you're the first cannibal in a group, you're better off and that's why the within group effect favors cannibalism and so the artificial selection ideas to emphasize the between group part by selecting whole chicken coops. I want to playfully suggest, well, maybe not playfully wrap seriously that chickens and researchers have a lot in common in this regard. We work in groups. We must compete, but we also must cooperate to get our work done. But the scale at which that competition cooperation happens has dramatically different effects on the phenotypes of the individuals and now of course in the chickens we're talking about genetically inherited phenotypes and that's not what I'm talking about in the case of students. I'm talking about culturally taught traditions and individual strategic adjustments that are responses to the structure and the structure which level rewards and punishments are applied at. And so we can think about the structural effects in science as being unintended consequences of the clumsy application of individual incentives for hiring and promotion and publication and grants and so on. That is that in many parts of academia, not all, but in many there's an over emphasis on short term individual evaluation, and this favors air quotes cannibalism. And of course all simultaneously limit scientific progress and this is by analogy to the chicken coop. The chickens all end up worse off because of cannibalism the system does not win. Even though the trade can spread because the individual dynamics allow it to. And this, again, evolutionary colleges understand effects like this this is a standard thing natural selection does not necessarily or even on average produce benefits for the population. So in the context of academia. I've been an academic for about 30 years now and I could tell stories. Since this is being recorded I'm not going to. But if we were in a live conference I would say buy me some beers later and I'll spill some tea. So when sort he is held in person at some point. The offers will stand. I will tell you all some stories off camera but there are bad actors there are also lots of good actors in science but they're bad actors there's these individual incentives can favor bullying manipulation exploitation and even theft of data and materials at times and I've I've personally known been adjacent to all of these things and and yes it chain for behavior and none of it is normative, but people get ahead through this and sometimes these these strategies are actually taught. I'm hesitant to tell stories here but again, some other time I'll tell some of you some stories. And of course the counter response to this is there are informal networks of gossip that arises resistance and I'm part of a number of informal networks as our I'm sure many of you and we do our best to work against this, but we're fighting an entire system where people get jobs and even some cases in some case, some cases, because of this bad behavior, and I'm just describing the situation. But I think I think the scale at which rewards are applied as a big part of this is that individuals, you know, the sort of stereotypical big men of science reward system favors this actually, and giving some thought to reform about sliding the scales and rewarding whole teams in units is is important. That said, there's no result in the system in which individuals don't compete it's just a question of what level we want the competition to be at what form we want it to take. And that said, if once you start rewarding teams you also can get different kinds of behavior within teams that's also not so nice because now, if someone in your team is is not doing their work it affects you materially and that can lead to other kinds of problems and so. I'm not saying that there's any kind of happy all ponies and rainbows world out there, but I think we can make marginal improvements. Okay, next metaphor. This is like a chicken coop it's also like a flock of feral sheep. And what I have in mind here is this famous, well it's famous to me I assume a lot of you know about it deserves to be very famous a study of the soy sheep from St. Kilda, the St. Kilda islands and these are our sheep that are, they're not even a flock because no one manages them they're free with dignity, right, unlike ordinary sheep. And there's a long term ecological study of the individuals in which they're measured their whole life histories and and their mortality and bursar are recorded and and as are the ecological conditions and this is a, this is a modern classic. These sorts of long term studies are some of the most important kind of work that we can do because you can't study evolution unless you've got the long term data. Right, the forces on an individual generational span to be very weak, but they can create big changes in the long term. So this is a really important study, and I want to highlight this, this one sort of paradox that has arisen from it is that the, the sheep are shrinking and fairly rapidly so what you see here in the plots on the right are lamb sizes and adult sizes in different years and kilograms, and you'll see that while there's fluctuations. There's been a loss of, you know about three kilograms for lambs since the 1980s, or for people my age we say since the Cold War, and adults have got have lost about the same amount of weight. This is happened, despite the fact that it's easy to show that larger individuals survive better, there's positive natural selection for being large for larger body size in this population. So why is the population getting smaller. And the answer is complicated, I'm going to give you the cartoon version of this, because I want to get on to the, the academic discussion part but a large part of the downward trend is due to earlier changes in lambs and changes in population regulation so mothers are having their lambs earlier, and then their daughters are having even smaller lambs as a result because they're, they're giving birth earlier and earlier and there's a nonlinear compensatory effect in the size of lambs for for young mothers in this in this population. Lots of things are influencing phenotypic change and most of them are more powerful than selection in the very long run selection will increase body size to some extent but the population is going to reach an equilibrium where selection doesn't get its way because it can't remove these developmental effects and the effects of population regulation. The quantitative version of this argument this is important because if we're going to be serious about policy analysis we need some logical framework to analyze policy and is the the age structure version of the price equation. I'm not going to spend really any time on this slide at all. I just want to fold out the terms of this for you so you take that previous price equation that came up with the chickens. And in an age structured population you've got a bunch of new effects that affect phenotypic change have to do with demography so like aging, just because individuals move from one age class to another this changes the phenotype. There's selection on survival there's individual plasticity, and there's a bunch of additional terms that have to do with heritability and how that works, and it's mediated by the environment trade off between fertility and size and selection on the and the really wonderful thing about this framework is it's complete it's axiomatic complete this equation contains all of the forces of demographic change. And so if you can measure any part of this you can compare its relative power to the other parts and you know you haven't missed anything that's changing the population. So this is a major scientific achievement to have a framework like this very few sciences get something like this that's axiomatically true. So in this particular case, as I mentioned before, the big thing that's driving down body size on the ecological timescale is this offspring mother difference mothers are giving birth to even smaller daughters who have even smaller daughters themselves. And this is having a big effect right and in the long run the body size equilibrium it's going to be some trade off between this developmental and population regulation effect and but selection wants. Okay. So that's sheep and I hope you like feral sheep to send another themselves because that's a fascinating study about how phenotypes change on ecological timescales and and how selection is not the whole story. In the social sciences. There's a whole bunch of similar issues to be appreciated and thinking about institutional change and the rates that it happens that you know, here's a nice textbook on the topic by Colin Cameron who's in a behavioral economist at Caltech and so here's just one example from the book but there's a bunch like this. The Stephen Jay Gould actually was why I chose this one cited this example in 1985 argues that baseball which is this very weird American sport, maybe some of you have heard of it. Yeah, it's mainly people drinking beer and swinging sticks. The swimming average is converged in the 20th century because of dynamic adjustments in field pitching and hitting. Some economists described this as an encouraging tale drawn from real life of how players learn to play equilibrium strategies and you can think of equilibrium strategies here is where the incentives will nudge individuals to. The point the camera makes over and over again in this book is that these incentives can operate very slowly in human societies, because people have to learn the equilibrium, and they're often not consciously aware of the incentives influencing them. In the case of baseball, it was the order of decades. So I bring this up. The reforms that we apply there are two things to say. The first is that they may have very slow effects, given some realistic demographic model of how academia works. And maybe that's just what we have to accept the second thing would be to say maybe we want to strategize about ways to make more rapid change by taking seriously that the population has to learn the equilibrium. So, again, this is a talk about the price equation now for some reason but there's a cultural version of the price equation to my colleagues Brett Bayhiem and Ryan Baldini have a paper from 2012 on this where they analyze changes in religious traditions in North America using a demographic decomposition composition of cultural change as norms for fertility. The main line Protestants versus evangelicals example, and this is the same price equation here it looks different but it's the same decomposition there's fertility there's differences between parents and offspring there's mortality there's immigration. This is now an open population I like the sheep. So there's immigration and immigration and all these things cause changes and people's beliefs and family sizes. So in this great longitudinal data set we can do the decomposition and show which forces dominate and so they analyze this church case and show that indeed it's it's it's not hard for small differences in fertility over the long run to lead to one church type to replace another. When we analyze academia now, and I promise I'm getting to the end of the talk. In academia, we need a model of a bespoke model of academia. And I'm going to give you a cartoonish version to spur your imagination here of what I mean and the goal here is to have a life history analysis of academic, so that I can draw out these structural forces and how they can swamp and make make the influence of incentives very slow in ecological time so to speak. So here's in classic life history style diagram. Here's the life history of an academic. So we start out as students s and then some of us survived to be doctors D and then some doctors survived the professors P, and then professors reproduced by creating students not literally physically reproduce but you know what I mean there's a cultural reproduction that goes on. And of course there's lots of details I'm leaving out here but this is this is an imagination exercise right. Also in these systems resources matter a lot. So there are students with stipends for example or with research budgets and students with stipends and research budgets may have a quite different probability of surviving to become a doctor than those who don't. The same holds for the other states in the system there are doctors with resources with grants, and they can, this can dramatically affect their probabilities to become professors and professors with money may produce different students or more students reproduce their ideas, publish more, and become a bigger impact on scientific beliefs than others. And of course at the same time individuals are exiting the system because there's a world outside of academia you may have heard, and it's there are many nice things about it. And a number of my students have gone on to quite happy lives outside of academia and I've completely supported them in that. And I myself am very happy in academia but there are exits. And those exits have a big effect on regulating the system to because, well obviously, often a professor has to retire before a doctor can become a professor, but it also on the other end for students. Some people can't become students. If professors already have students, right there are limits on these population limits in different parts of the system. So when, when we end up people do policy analysis of different systems, they use diagrams like this and try to think it through in the mox pond society, for example, we're trying to create a tenure track system at the institutes which doesn't traditionally exist. And we have flow diagrams like this where we're trying to figure out how many people can you push through such a system giving given retirement rates and the sizes of budgets and so on to have a really responsible demographic analysis of how such a program might work. And what you can do, though, just purely intellectually without analyzing a particular policies just try to understand how these systems behave in general and this is the general thing about complex systems analysis which is big in evolution ecology is you do thought experiments with these simple models to say you delete some arrows and make some others more important and observe how the system evolves. And this help gives you intuitions from the simple systems that you can apply to real world cases. And also, by the way, why, why I encourage you to think about chickens and feral sheep is because, even though researchers aren't exactly like those things in complex systems you have to take isolated lessons from from from analogy to draw your attention to your particular system and then you'll do the bespoke analysis on your system but you get it, you become aware of these sorts of dynamics by studying general systems. The top of this demography of course is the evolution of scientific beliefs itself, which I'm on. This is the only slide I have on that. Paul and I have a paper in 2015 in which we try to model the influence of publication bias on the cycle of investigation and hypothesis formation. And this is a very cartoonish model. There was there's very little literature on this actually, even though it's such an important determinant of how our beliefs form and which theories float to the top, or sink to the bottom. Okay, I'm conscious of the time and I'm going to be done here. But there's a saying in the social sciences that demography is destiny. And it's a deliberate exaggeration obviously it's not destiny is just say demographic forces are incredibly powerful. And if you're unaware of them you can be badly surprised and academia is a system with many bottlenecks in sequence, and those bottlenecks create powerful demographic forces, and we need to think about that just as hard as we think about explicit problems because they transform the population and our beliefs in powerful ways. And so just some quick examples to stimulate your imagination in a rapidly growing field it can be very hard to police. The research that's going on because there's just too much bullshit being produced at too high a rate. I'm not going to name any fields but take your favorite example. And this this selects for favors rapid training as transitions from student to doctor because that gets you into slots. And so any research tradition which which graduates students rapidly will have a head up in this in this rapidly growing field, and as a consequence scientific beliefs can experience lots of drift because the effective population size can be very small, and also draft in the sense of genetic draft which is hitchhiking here it's cultural cultural hitchhiking or cultural tradition or statistical procedure or way of measuring something that becomes common just because of who started the field. And again I'm not going to name any fields but I'm sure you can pick your favorite example. This is, I think this sort of thing is really common in the sciences and we should do something about it, because it's bad. Okay, in shrinking or stable fields you have other dynamics which are also very powerful. If you have very slow replacement of professors, this dramatically changes how all the competition works. You necessarily get a slower transition from doctors to professors. This means you need ways for individuals to have happy productive lives when they don't have secure employment and this is a major struggle, especially in Germany. Right now this is a big struggle. Okay, last two things and then I'll end the talk. I want to give you two examples of effects of bottlenecks that are, I think, not as intuitive as the other things I mentioned and so I want to mention them before I end. And the first is there's this very powerful thing that happens in almost all bottlenecks all all strong selection scenarios, which is called the selection of distortion effect and it's well understood by statisticians, but in my experience, scientists have not heard of it. And the second is that bottlenecks make luck the most powerful force inside a system. So, here's a paper that was published just this year in in one of the magazines I don't know science and nature I forget which one. So, looking at findings that were later replicated or not, and asking up until that replication attempt happened. How well cited the result was so if you just look at panel a for example these were studies that were published in nature and science the black trend are findings that were later not replicated. And the vertical axis is citation rate yearly citation count, and the horizontal axis is year, and the blue trend are findings published in the same journals that were later replicated. As you can see and this is also found in economics and in psychology. Findings that were later not replicated were cited more before the replication attempt failed. Okay, so this one impression you might get from this is that there's two kinds of science going on here there, there are people who are doing their q rp's and p hacking and going for the glamorous nature science pub and those that's the black trend. And those are unreliable results. And then there are the, the good drones, right, people like me who published more slowly and try to get everything right and put everything on GitHub and so on and we want to be the blue trend. Okay, now we don't get published in nature and science nearly as often to be perfectly honest, but we want to be the blue trend. And that may be true, but I think a lot of this is just a selection distortion effect that comes from peer review. So, bear with me very quickly for for an example. This is an example that's in my textbook by the way on page 162 and I give the code to reproduce this thought experiment. Imagine there's 200 papers or grant proposals, the example works fine for for either in which there's no correlation between two features of this of this work and the on the horizontal axis newsworthiness. What I mean by newsworthiness is the potential broader impact of the work how excited other scientists in the public would get about it. The press sensation aspect of the work. And that's not necessarily a bad thing. We want science that has impact right so newsworthiness is not a bad thing. Some sciences newsworthy some science isn't and there's no judgment either way. It's just a different aspect society often cares about newsworthiness though. And then there's the other axis which is trustworthy this which is the reliability or any of the ours from my list earlier reliability reproducibility. So now, imagine, we create some composite score subjective score which is some additive combination of these two axes, and then we select the top 10% as I've done here in red. And a consequence of this type of selection in any kind of system and again this is a called a widely appreciated effect in statistics is that this induces a negative correlation between the features post selection. And that is selection, distorts the associations among the features, and often very powerfully so and the, and the more intense the bottleneck that is the smaller number of items that are selected here proposals or papers, or candidates in the case of jobs, the more intense the distortion. So post selection in this example you get a correlation of minus point seven seven. So the code to reproduce this is in the book you can you can run it a bunch of times and verify that I'm not I haven't picked exactly the example that shows the effect I want. So, the point here is that all of the bottlenecks whether it's hiring or grant review, or publication will distort correlations like this and create these negative correlations between things like how trustworthy work is and how much attention it gets, but it's not really a consequence of cheating, but it is something we still want to deal with. And I hope that point comes across. Last example, I promise kitty wakes. So, Robin Snyder is one of my favorites theoretical biologist and she has a pair of really nice papers with her colleagues on the decomposing and quantifying the roles of luck and pluck. So, you know, stochastic effects and quality effects in ecological dynamics, and these are great papers. I really encourage you to take a look at them, even if you don't want to go through the math there's lots of really they're really well written Robin always writes really well. And so, here's from this paper that came out this year time and chance where they, they take a number of data sets and they do the decomposition to look at the importance at different ages of luck that is just being lucky enough to make it to the next life history stage versus quality differences on lifetime reproductive success in natural breeding population so here's just one of the examples from from kitty wakes. I think these are black footed kitty wakes, and the graph what you're seeing is age kitty wakes can live a really long time like lots of birds right most of them don't. And then the vertical axis is the contribution to the variance in lifetime reproductive success of the black trend is luck. And this is the ability that is who survives, and then the red trend is quality. And what you're seeing here is that the life history of these birds is dominated by stochastic effects unrelated to their phenotypes and that's because it's very hard to survive fast like age five or seven. And in life histories like this which are quite common selection has an effect in the long term but it's incredibly weak. And so we understand these systems as being mainly determined by luck. And I think academia is unfortunately structured like kitty wakes. You've got big populations with very intense sequential bottlenecks. And those bottlenecks are, I'm sorry to say, often regulated by things other than quality differences. Here's the one Lord of the Rings joke for Karina. So, and this applies both the researchers and our theories, right that we passed through a lot of type bottlenecks and promotion funding and citation. There's competition for attention and for the money that will generate the work in the first place. And lots of things it's not that quality has no effect on this competition, but such a small number of individuals and ideas can make it through the bottlenecks and lots of other things like network effects who you know which school you went to your cultural background. All of these inequalities and forms of passive instructional discrimination can have huge effects on the system and I call these things luck. This is a bit of a euphemism. Of course, it's a deterministic universe. But what what I mean by that is it's not the stuff that we want the system to select for. And I think this is a big, almost moral crisis in the structure of these systems. And I'm not saying everybody can get promoted, but I'm saying we should we should work hard to govern these bottlenecks so that they they produce what we want them to produce and not produce massive distortions. Let me let me close. I know that there's been a ton of stuff and I've spoken really quickly and I've spent the last hour talking here and and we need to socialize some more. So I give it given some examples about hockey players and chickens and churches and sheep and and drawn connection to researchers and all of this stuff I think is super relevant to ongoing debates. And I guess reform movement is hot now and every country's talking about it. I get emails from the US government asking me to be on panels about about scientific responsibility and and in Germany there's this big discussion which I put on the slide here right now about German employee employment law and how it creates a precarious issue for researchers. And unfortunately the government defends it and this has become a very hot topic now and maybe the law will get repealed, but currently this is this is something that the ideas in this talk speak directly to where we where the legal system creates competition actually distorts the kind of research we get. And saying things are hard sometimes can seem like a defensive the status quo so I want to conclude by saying that's not my goal at all. Small changes may be expected to have very small effects and take decades to filter through the system because of the roles of bottlenecks and the long life histories of academics and and legal systems. But some big changes that are quite accessible to us I think are really worth it so I just want to highlight this paper from he's been in bright is peer review a good idea and they conclude no. It's a really good paper, and I just wanted to suggest that you take a look at it. Okay, let's stop here. Where do we go so I'm calling for is serious analysis of policy suggestions and not just to say we share an ideology about how we want things to be and we should adopt incentives that support that ideology that's that's good but getting there from here could be quite hard and it may take a very long time. That is we're trying to construct a niche for good science to happen in Avenue of course is to do the science and technology studies approach and to study reform movements in other places and try to see which lessons can be applied for us. And there are successful reform movements laboratory protocols in the 20th century especially were dramatically revised and now many laboratories use these laboratory information management systems limb systems, which are a form of version control and data systems, which is very successful. And of course software development. There was a revolution in the late 20th century in organizing teamwork in software. You can think of it as the GitHub revolution right the get revolution. And then your idea, obviously, people listening here are the future of the field, and your ideas and the most important is a pipetting by mouth I should say there was a joke on this slide pipetting by mouth people used to pipet by mouth. Do not do this. This is a very bad idea. Much of the activity that goes on in scientific research outside of laboratories are in laboratories is the metaphorical equivalent of pipetting by mouth, and we want to move our culture away from those things. Okay. Thank you for your indulgence for a very weird talk. If you've enjoyed this there's another weird talk called sciences amateur software development that you can find on YouTube just Google it. And with that, I will stop sharing my screen and I'd love to hear what