 Hello! I'm very conscious of wanting to be outside of it here, and so I'm going to set myself up to this physics, yeah. No, it's, I'm excited. Okay, we're going to get done with, I'm going to give you an example of a mixture model, a really useful one that'll set up your homework, which is already up on the website. And I want to get started with the conceptual part of multi-level models. It gives us a little bit of a head start so we can do more quality time with the actual processing of them in the next two weeks. Because we only got two weeks, so it's going to be all multi-level models. So that's the agenda, and that is exciting to me, and I know to some of you too, you're like, when is the class begin? Right, because you have some of this stuff, and you're like ready to put multi-level stuff. It's coming, but it's just I'm paving the way, and it's good to have, it's good to have a kind of methodological introduction to the more basic forms of underpinning. Okay. Yes. The question was, you're going to give feedback on the homework. First answer, no. Second answer. Well, it's a lot of you, and I do pretty detailed solution sets in that, but if any of you have particular concerns about something, like you do the comparisons between your solutions and the others, and you're wondering, like, oh, is this a big deal? But then, by all means, contact me, and I'll try to pick you in. Lately, and some of you have been trying to meet with me, and you know, my calendar is incredibly impacted right now, but as February ends, it will get better. I know, it's just really annoying, but we don't know anything about it. That's probably not true, because Paul, when Paul's been annoyed, he's been emailing people individually and saying, give dog, don't do that anymore. So, if you haven't received a grown-up Paul email, I know some of you have, because it usually blinds you, sees me, huh? But, in general, you guys are doing a really great homework. A lot of you are using Markdown, which is wonderful, and I haven't pushed that just because I learned to reduce cognitive load, but I think it's a great Markdown script because it self-documents for you later on. In Future U, there were some of you in my office hours earlier today, I was telling you this. Future U will not know the things that Current U does, so Future U will want to go back in time and make an interview with Current U. And your notes, when you really document your model code, as you do that for Future U, you're doing Future U a favor. So, my model notes are like that for my publish papers, like there's just huge amounts of comments in them, and Markdown's nice because it sort of encourages that documentation and makes it natural and freeform. Is Markdown a thing, or is it just sort of a pound of comments? Markdown's a thing. Well, that's a weird thing. Everything's a thing. What's that? What's the word people for Philadelphia use for thing? John. Well, we hit what? Yeah, exactly. So, it's just being a thing, right? That's it. Anyway, so it's more than a thing. It's a system. It's a system of, anyway, Google it, Markdown, or Markdown, Google it. And I know there's some real pros of it in the class, so they might help you with it if you want. I'm not pushing it on you necessarily, but it really does help increase your sanity reservoir sometimes. Do you like, do you comment within Markdown's better than just commenting on in- Well, it makes it pretty your document. So, it's like you're writing, I mean, you can write a published paper in Markdown, and it has all the R code in it, and it'll get executed as you rerun the document. So, it's a great thing for helping document and replicate analyses. And then your colleagues, again, exactly. So, it's a good system, and it's caught on really fast for that reason. And then when your colleagues want to know how you did it, you send them the source files with the whole paper, and it's got all R code in there. It's great. Yeah, that said, when you're running, when you're running Markout Chain models that, you know, take eight days to sample, and you don't put that in your Markdown. But so lately, I haven't been using it very much for that reason, because those are the models I've been running. They've literally, I just waited two weeks for a model to finish mixing. But, hey, you know, there are exceptions to all workflows. Anyway, other questions for you going here? I know the people who aren't in the class and have been watching the lectures love these weird stories at the beginning, right? People have babies and all kinds of stuff happens. Okay, what are we actually doing? Today, we're going to start with mixtures, and mixtures are also hybrid models. They have multiple stochastic processes, as you might think of them as likelihood functions. But the measurements that we have available have mixed those different processes together. I'll say this again. There are models that contain more than one likelihood function, so there's one more than one stochastic process that's generating observations in the world. But our ability to measure those processes has produced a mixture of those processes, so we can't easily disentangle them just from the raw data. Now, I'll give you an example of how that happens. One of the most common cases in which this happens, and we'll inspect this next week, is the case of over-dispersed counts. So, remember when I introduced count variables like the binomial and the Poisson? I said that for all counts, the variance scales with the mean, the greater the magnitude of it, the greater the variance in those counts. And that arises naturally from the maximum entropy nature of count data. When we observe counts in the real world, and we make a count variable, typically that variance that we measure, even after conditioning of all the available predictors, is in excess of the expected variation given the mean. And that is often called over-dispersion. The counts are dispersed. They're wider than we'd expect if they were all from the same homogenous process. And that's because nature has heterogenous processes that generate variation underneath it. And you can very usefully model over-dispersion with mixture models. And the most classic ones are the beta binomial and gamma Poisson. And there are worked examples of those at the end of the count chapter. I'm not going to lecture on them. And I used to. And I'm not going to lecture on them partly because they're not nearly as useful as they used to be. Because now we can do exactly the same thing with multi-level models. So I'd rather punt on that until next week and give you a multi-level example. That will help broaden your understanding of what you can do with multi-level models. The multi-level models are our way to get heterogeneity into any level of the process that's generating the data. And that's what these sorts of mixtures are. Beta binomial and gamma Poisson used to be the only game in town because we didn't used to have Markov chains. So we could fit arbitrary multi-level models in. And the beta binomial and gamma Poisson have this conjugate structure that makes them easy. There are integrals that can be closed analytically. And then it's easy. I say something about this in the notes if you're interested. You'll come across people using them. They're still very useful models. There's something pretty tricky to fit the data though. And ironically, complicated multi-level models are easier to fit the data than those in my experience. It's a weird thing about them. So we're going to spend time instead in class on zero-inflated mixtures, which are incredibly common. And I think more common than people notice. So I want to draw your attention to them in a toy example. And then your homework has a parks management example. I call the homework problem parks an estimation. So here's the toy example in class. And this will be an example that also lets me re-emphasize to you the value of simulating dummy data to help you understand models. So we're going to do the model forward and get data out of it. And then we're going to do it backwards and get parameters back out of the data. This is a great thing when you're debugging your code to do. To reassure yourself that you have written the model right. So here's the setup. I want to imagine you're a monastery tycoon. You own a number of monasteries. These things used to be monastery tycoons. Prior to the printing press, monasteries were the primary way that you got the printed word out to people. You had monks and they toiled and copied manuscripts by hand. And they practiced their penmanship very well. And different monasteries have different productivities. And you're a monastery tycoon. You need to figure out how hard your monks are working. Determines the market valuation of your monastery and other things. So, monks copy manuscripts. The problem is they have sellers full of delicious cider. So they also get drunk. And throw wild parties sometimes during the week. And this is all true from the historical period. So the data we're going to be looking at, the outcome variable of interest. If you evaluate the economic performance of some particular monastery of yours that you're interested in, is the number of manuscripts completed on each day? There's this log book. And as manuscripts are completed and shipped out, someone writes down in the book how many were completed on that day. So it's a count. And this will be a plus on variable. You might think, right? Because there are the manuscripts. On many days, no manuscripts are completed because it takes a number of days by hand to complete a 400 page illuminated manuscript. It could take a while. So there are many days where none are completed. On many days you get like four. That's like a day where everything lines up, right? But there are lots of monks, many, many more monks working in the monastery than manuscripts get completed on a given day. Because it takes much longer than... Makes sense? So very large number of trials, folks, working on manuscripts, and a very small probability of completion on a given day by no means drinks to a plus on density does good work here. The problem is there's heterogeneity in the process because they're not working on every day. Some days the monastery kind of shuts down and drinks. And they go on a bender. And on those days there are zeros. They don't produce any manuscripts. But even on days when they work, they sometimes produce zeros. So this is the kind of data that we call zero-inflated. There's more than one process that produces zeros in the data. And you can't tell just from the data. Or can you? And that's the thing we're going to look at. So let me... Let's break out the analytical structure of this a little bit. And this is called a zero-inflated plus on process. And this is the most common, I think, zero-inflated model. You can also zero-inflate binomials or any counts you can win the... Well, I'll talk at the end about... There's a whole family of stuff called hurdles, which are related to this as well. There's a hidden state here, the generating of the data. And that hidden state is whether the monks were drunk or sober on that day. Now you can think of this as whether they were partying or working. And so there's an initial probability. You can look at the diagram on the right-hand side of this slide. P of the time they drink, right? And then you always observe a zero. Y's are outcome variable, are counts, are plus on count. One minus P of time they work. But there's some chance that you get a zero even when they work. And then some chance you get a number greater than zero. So when you observe a number greater than zero, you know they weren't partying on that day. But when you observe a zero, you don't know. And our challenge is to estimate how much partying the monks are doing. And it turns out we can do it when we write down the likelihood in a principled way following this diagram. So that's what we're going to do. We're going to construct a new likelihood function that is a mixer likelihood. There are two stochastic processes in it. The first is binomials. Are they partying or not? There's a chance P, they party. And the additional on not partying, there's a plus on process that generates complete manuscripts. And what we observe is a mixture of those two things. And at the end we've lost the partying dice or coin flip for partying. You can imagine the monks have a wooden coin on one side as a big frothy sign on the other side as a quill. And they flip it every morning. And the question is what's the weighting on this coin, right? I should say some of you know this. It doesn't matter how heavy you want to tell you that. You can't buy us a flip coin. You've got to spin it. Spin it. It'll be biased. It's flipping it. It doesn't matter how heavy it is. Try to put a big board of gum on the side of a quarter and flip it. It won't matter. So we're going to build a new likelihood. Let me walk you through the idea. We're going to simulate this data later. And it's going to look like this. So I just want to foreshadow for you what the data looks like. Manuscripts completed across the horizontal and just account the frequency on the vertical to those values over... I forget how many days I simulated this. It'll come out later in the script. What I've highlighted for you is what you wouldn't normally be able to see is that there are some number of zeros. In this case all of the ones that are shaded in blue are the drunken zeros. The zeros that arose from part of the game days. And normally you wouldn't be able to distinguish them but they arise from the part of the diagram that I put in the blue box over there on the right. Does this make some sense so far? So let's build this up now. We can derive the analytical form of the likelihood function for this. Now I have, of course, written a function in the rethinking library that already does this for you. But I want you...it's important to me for you guys to understand the magic here. Right? Or the lack of magic. Whatever it is. The guts of the golem here are not that complicated. And these mixture likelihoods are really just hobbled together from the logic of your story about how the data are born. Remember early on in the course I said a really pragmatic way to work towards basic computation is you want to tell yourself a data story. What's a generating story of the data? If you can make that story into an algorithm you can write it in backwards order and you have a statistical model that can recover the parameters from the data generating process. And that psychological procedure of thinking about the data forward and then writing the model backwards nearly always works. It's a very powerful way to approach it. And think about writing down the likelihood function from it. So let's go forward for a second and then we'll write down likelihoods analytical likelihoods that give us the probabilities of zeros and non-zeros. So the first thing that happens is our binomial process. This is P at the time they drink. 1 minus P at the time they don't. So P at the time we're guaranteed to observe a zero. 1 minus P at the time there'll be some Poisson process that kicks in. Is this the quills paper? Go in. Some of the time that Poisson process produces a zero and it produces it e to the minus lambda of the time. You're asking where does that come from? That's the Poisson likelihood function. So if you Google Poisson distribution and you go to the Wikipedia page you will find the mathematical formulas for the Poisson distribution and if you set the outcome to zero it reduces to this really nice thing. It's just e to the minus rate of the process. That's the probability of a zero out of a Poisson process always. So that's where that comes from. Conditional on their working the probability of a zero is e to the minus likelihood. And then there's some other probability if you observe an N instead the probability of it is that thing which is just the Poisson likelihood. That's what that is. Where does that come from? If you take the binomial likelihood and you take the limit of one over infinity that's what you get. So now we can think about writing down conditional on observing a zero there are two ways it can happen and we have to account for them both. So now think about it this way you've got your data set there's some case where a zero has appeared what's the likelihood of a zero what's the account for all the pathways in this diagram by which a zero can arise and we have to use them all and you may remember from probability theory at the beginning of this course or in some former course that when things happen together in probability theory we multiply them and when things are alternative they happen separately we add them so we're going to have an addition in here because there are two pathways to get a zero so there's a four that appears. So the first thing that can happen is P at a time they drink and we're guaranteed to get a zero so the probability of a zero conditional are P or that's why we add plus one minus P at a time they don't drink and the Poisson process produces a zero which is E to the minus lambda. Make sense? Yeah this is all probability theory is it's just and and or and write down all the things that can happen figure out all the ways they could happen count up all the ways that any particular event can arise that's all probability theory is it's just counting it's a compressed form of counting Does this make sense so far? Next step, what if it's not a zero conditional on not observing a zero the chance is one minus P because we have to get down the right hand side of this tree times the probability of any value in greater than zero which is that expression, lambda to the n times E to the minus lambda over n factorial Make sense? So we get a likelihood function that has these two terms in it conditional on observing a zero the likelihood is up there on the left hand side of the slide conditional on observing something greater than zero it's that thing over there Notice that even though when you observe something greater than zero you know it's a Poisson process you still have to multiply it by one minus P because they worked and managed to produce some manuscripts so there's a multiplication Make sense? I'm not expecting you guys to have a calculation that you have to derive the likelihood although that would be fun for some of us but I'm not going to expect you to do that in this course it's good to know something of the logic and eventually in your careers you may have a case like this where you need to cobble this together and whether you, I think all of your people are doing it yourself because I see you do harder things in this course than this it's just that this is unfamiliar and you haven't done it before so I want to expose you so that you know it's within not that hard like I said you've done much harder things already so back how do we express a zero inflated Poisson model now there are two parameters that describe the zero inflated Poisson distribution which is a Poisson distribution salted with some extra zeros it's got a big excess of zeros that's why it's called zero inflation and there are two parameters that describe its shape there's one parameter, lambda which is the rate for the average number of minutes per unit time and this is mixed with a binomial distribution or Bernoulli trial at the start and its parameter is p so the zero inflated Poisson distribution is a function of both of those parameters so we have two linear models if you so choose you don't have to you can just ask to make p and lambda but if you have predictors the probability of drinking and the rate of manuscript production conditional on then you need two link functions and two linear models and the linear models can be completely different from one another and they have different parameters it's up to you so the example I've given here there are two different intercepts two different beta coefficients and then a common predictor x whatever that is like how sunny it was outside whether it's inside the drink or not whether it's in the cellar x could be that but they don't have to be given your background knowledge of the system there can be different predictors in these different processes it could be something completely different to determine whether they party or not compared to how fast they produce manuscripts when they're working could easily be the case I had an older example where it was grad students producing manuscripts I thought that was too close to home that's also zero in place I have a data set on that curious sometimes does this make sense yeah good question why is the second one log because that's the Poisson rate and what we want to do is constrain it to be positive lambda needs to be positive so the customary link when you want to constrain a parameter to the positive reels is to use the logarithm because if you exponentiate that linear model yeah so we're using the conventional links from the binomial model and the Poisson model they're still in here but both processes are in the same model now okay so let's simulate some data now and there's a whole bunch of classic paintings of drunken monks it's really great go to google image search sometimes put in drunken monk image tons of great stuff and it's not all Jackie Chan movies some of it is I think it's very important as your models get more complicated and you start to feel yourself being nervous about your conceptualization of the model structure to simulate data out of the process to make sure you understand what's implied and if you can recover the parameter values from your simulation at least get the neighborhood of them then that guarantees that your model is working right and that you understand what the model implies about the data this is a great thing when you're developing new models this is absolutely essential for new model types or fancy codes so for these models I just finished a two week model fit you can believe that I started it off this way made sure it worked on simulated data first before the two week run on real data in my code before I knew it worked and how did I because I had fake data where I knew the truth getting in there and you converge on it and then you trust your software and you can run it on the real data this makes some sense this is very important the models in this class are mainly pretty simple but still simple is in the eye of the beholder and you get something extra out of the forward simulation approach so I'm going to show you that here because it's really simple and it will give you an example you can scaffold off of in later work so we're going to simulate dummy data I like to call it we're going to recover the estimates and hopefully this will increase understanding of the model what we're not going to do in the example here but I encourage you to try at home is to try parameter combinations that are extreme and you can see that sometimes you really can't recover the process because they're sealing in floor effects and these nonlinear regressions and that removes information from the data about process this pushes things in really extreme directions you can't figure out there'll be just a bunch of parameter values that are consistent with the data and absent a psychic friend you can't recover it and that's important to know now the nice thing about that is the Bayesian posterior will tell you that it'll be wide over some range because it's telling you the relative number of ways that each parameter value could have generated the data conditional on the model so it isn't going to lie to you and give you false precision but you shouldn't expect to always be able to recover the data generated process in the nonlinear model it's just the way the world is it's called the generalized inverse problem if you might come across that in the physics literature it's a cool fact about doing science I think it makes it fun it does because there's a problem to solve you need to find another way of looking at the data some other kind of data description on a different scale to solve those problems okay so let's simulate some manuscripts so this code I want to run through it real quick and just so you guys understand how easy this can be first we just define the probability of drinking we define the fixed true parameter values of the process the probability of drinking is 0.2 so on 20% of the days the monks drink I'm not sure historically what it was like but if I were a monk I would drink a lot that's all I can say um just but how do you get to 0.2 I just made it up you want to try different values obviously you just kind of like pop in yeah you try different values because you're validating your code and you're trying to understand the model that makes sense this is not a hypothesis but it was really 0.2 and you're not going to publish these data right this is about understanding it to make sure the code works I hesitate to say that value doesn't matter because again for extreme values really high values near 1 and really low values near 0 you'll see that the estimation problem is a lot harder um it is for 0.2 it's pretty easy I'll show you an example but you make up the example because it's a form of intellectual play that makes sense it's not cheating right and unless you publish this right which you definitely don't do I mean this is kind of the anyway to start talking about fraudsters but I was going to say fraudsters usually luckily aren't sophisticated enough to know how to do this and then they do things like everything's perfectly linear in their fake data it's like come on people give me a challenge here the data forensics community is like whatever like those social psychologists that made everything like a perfect straight line something it was like come on you're not even trying to drink this what is a red number generator it's an issue it's fortunate anyway you guys are honest so I'm not worried so in is there's like nervous giggling in the crowd so well you're going to publish all your scripts everything will be audited so you're the new generation where everything's open methods and replicable absolutely I will haunt you I will haunt you I will haunt you a drunken Irish ghost following you into the professor hood so alright where was I sample one year of production oh yeah rate of work on average one manuscript a day that's the Poisson rate right there we're going to do a whole year of simulations just because so the first stochastic process the one that's late in the real data that we don't get to observe we don't have data on this you can imagine getting data on this but if you add monetary tycoon you only have that logbook the months that tried to hide this data from you that we were about to recover it because we were stat hot shots so they didn't know that so do they drink or not and you just put probably a drink in there drink now holds a bunch of zero and one we can indicate whether they got drunk on that day or not then we pass that drink variable right into our simulation of the Poisson distribution and notice that formula looks like the likelihood that we wrote it for is one minus the probability of drinking whether they actually can now indicate whether they really drank or not times a random Poisson number in of them with an average value from rate underscore work why holds zero inflated counts with me it's really all it is to it does this help you see what's actually being assumed so now let's look at the model this model is easier I'm not going to put any predictors into it but these are just linear models so all your previous tricks fine you can put interactions in there whatever you want to do but sky's the limit let your science decide and you put this into map exactly as you might expect dzi plus is a likelihood function, a density function that I programmed into the rethinking package there's a box in the notes where I do the dissection of it and show you what it's got to look like in case you ever want to write your own likelihood function there's a scaffold in there for you to see how it's done it's not that hard but it's one of those overthinking boxes that I say you don't need to read until the second time through the book does this make any questions about this the only new thing is that dzi plus is there alright when you look at the estimates exactly as you might expect here's just looking at the pracy output in the middle of the slide we get an estimate ap is the log odds of they're being drunk it's minus 1.39 let's call that minus 1.4 now it's log odds so to get it back on the probability scale you plug it into logistic right so I do that down here in the bottom of the slide logistic of minus 1.39 is 0.1994 which is close to 0.2 which was the data generating process there's a lot of days here 365 days a day you can recover the process now for lambda AL is the log scale average manuscript production per day it ends up being very close to 0.05 so we exponentiate it to get it back on the count scale because of log length you can use the inverse length and that is nearly one which is what we assumed about the process so it works makes sense about the inverse length going log scale to the probability scale we're plotting scale to the probability scale and then plot the probability of success from the logic scale to plot the probability of success yes you gotta undo the logic transform so logistic it but you could do things in log odds you can do that it depends you really get probability well there's some graphs where we look at the parameter scale and it's on log odds then there are other graphs the posterior prediction checks are all on probability scale again it's a horoscopic advice time so in a particular context I could sit down with you and we could figure out what's most useful here there are cases I was collaborating with a lab in Lansing they may be Hello Isis if you're listening to this they work on kicks I'm a sucker for creepy things no it was a cool staff problem it's about chick questing this is what they call it chicks climb up on things and they wait for warm objects to pass by like jotters and they jump on them so there's this questing behavior where they crawl up on things but they get desiccated really quickly so they gotta go back down and leave litter and chill for a while and get their moisture back so there's this questing behavior and so they have a bunch of experiments field trials with questing behavior of ticks most of the time the ticks are in the leaf litter so you want to estimate the probability that they're questing on any particular cross section it's really low so the probability scale is basically on the bottom axis always but there are big differences across treatments on the log odds scale and it has huge consequences for the system like probably a jogger gets Lyme disease and so it's a cool problem where in that case I looked at everything on the log odds scale because you actually see differences and they're very reliable differences ticks are savvy good ants are little spiders but yeah anyway does that help? okay alright so yes I think I'm done with this slide right you guys tell me I forgot where I was before I started thinking about ticks again I'll wear my tick shirt next week I had this tick shirt they gave me called go for blood anyway so other mixtures so let me just review back to this you can also do zero inflated binomial and there's a density in rethinking to do that as well you can derive it the same way it's just the false on gets swapped out with the binomial it's the same idea there's a whole bunch of hurdle models that combine some processes generate zeros with a process if it isn't a failure you get a continuous reading and a distribution or an exponential those are often called hurdle models they arise a lot in biochemistry because you're trying to detect the concentration of something in a solution and your assay doesn't work until the concentration is above a certain hurdle so you get zero inflation because there are these low concentrations where it's there and they all read exactly to zero because the assay just shows up to zero in those cases this is a common bit in science you know what I'm talking about I published a paper two years ago I think on human data where the hurdle model was useful because it was foraging trips so guys go out hunting in the Paraglain forest and since it's been a time they bring back nothing that's a zero harvest is what it's called sometimes they bring back a hallowed monkey and the record is the kilograms of meat that they brought back so that's a it's not a zero if you could call it a kilogram model but it turned out to be a zero inflated gamma model which you can construct the same way it just deals with the fact that there are two processes that are producing the outcome did you get anything and then how big it is there's two processes that go on there so there are also two processes that produce an outcome let's say environmental processes like nutrition and strictly genetic processes like genes those are two processes but you're saying they're not mixtures those look like regular linear models so what's the there's this issue of abstraction here when I say different stochastic processes at the level you're representing it in the usual thing we'll say there's a bunch of small additive effects and they combine developmentally to produce height it's not a mixture of effects but we use the Gaussian because we think all these small things are adding or we just use the maximum entropy thing that we don't know anything about the processes but we do know it's it's actually unconstraining the continuous does that help the vocabulary doesn't really do justice to all the choices you have to make in this stuff sometimes I think I've seen a number of times where these type of models should be used but the normal binomial or normal great question I should have I should have made a big deal out of that in an earlier slide so the question was lots of times we see cases where models like this should be used but people just use ordinary binomial or normal so what's the consequence the consequence is that you will underestimate the rate so for the monks if you assume that they work every day and there's zero inflation you'll get a lander that's smaller you'll underestimate their productivity when they actually decide to work unconditional on whether they're drinking or not their effective rate of production is low and that's what you'd be estimating if you just treated it like a Poisson but that's you're getting the wrong inference right because if you could actually crack the whip and make them work every day they'd be a lot more productive if that makes sense so that could be a big deal and your homework is meant to be a pragmatic instead it's a park management data set that is about this, it's about fish extraction from a national park and the amusement park actually tries to get fish but you can't rely upon their testimony about that so anyway so that might help you think through it a little bit but it's this issue of getting a more accurate measure of the actual rate of extraction just to clarify on the genetics if you knew something about the genetic process and how it was monadative like who knew if this gene and it has this dominant relationship they could use this kind of model maybe I need to see again I'm stuck in horoscopic land up here and I'm the astrologer physical astrologer I think often if you want to model development you can do way better than just saying the outcomes Gaussian absolutely especially if you're so I know I have some colleagues that work on C. elegans development and the stuff that they do is scaffolded like these models you have different stochastic processes that they go with genes per month because we know everything about the genome of C. elegans so definitely some hope to do stuff like this if you know there's a gene that turns on early in ontogeny and it can go different ways depending upon what happens then that'll have consequences for the distribution of the later developmental processes so yeah I can imagine I guess I just didn't imagine things like that but I've never worked on anything like that yes and I said continuous mixtures are important we're going to focus we're going to work on multi-level models to do those okay remainder of the day