 So thanks for that, it's a nice introduction. And I thank you all for being here, you know, the first day after Thanksgiving holiday. As daddy said, you know, I was in Southern California, I attended UCLA, and it's very, very nice to be back to Southern California again. And I was just talking to daddy that, you know, there are some people talking about, you know, or I'm being boring, but, you know, you only know the nice thing about Southern California after you get out of here, as I did. So today I'm going to talk about multi-level latent class regression of stages change for multiple health behaviors. This is a work I have been start doing when I'm collaborating with centers for American Indian and Alaska native health when I was in Colorado. And then, you know, I keep moving on, I keep working on it when I move to Texas. So here's the outline of my talk. The after a brief introduction to introduce the motivational problem of this project, I will give a brief introduction of latent class analysis. For those of you who are not familiar with this statistical method. And then I'm going to introduce the modeling and estimation for multi-level latent class analysis. And then I will present some simulation results as well as a real data application analysis results. And I'll conclude with some discussion points. So just a brief introduction to give you the concepts of stage of change model and some concepts of multiple behavior interventions. Proposed by Prokazka and his colleagues in the 1970s, the essential ideas of the stages for change model is states that behavioral change does not happen in one step. People tend to progress through different stages on their way to successful change as shown in the diagram on the right panel. And the stages of change model theory propose that intervention should be tailored for participants who are at different stages. This model has been applied to a broad range of behaviors including smoking, weight loss, exercise, and eating habits. So more specifically, the stages of change model theory propose that the process of changing for behavior usually has five different stages. So the first stage is so-called pre-contemplation stage at which the individuals are not intent to change in the foreseeable future. And then the next stage is the contemplation stage at which the individuals acknowledge the problem but they are not prepared to change their behaviors at present yet. However, they are seriously thinking about changing in the near future. And then the individuals at the preparation stage include those who are actively considering changing their behaviors in the immediate future. And the individuals at the action stage have already made behavioral change recently but the changes are not well established yet. And the last stage is usually called the maintaining stage which include the people who have changed their behaviors for extended periods and they are working to sustain the change currently. So nowadays, effective lifestyle interventions to prevent chronic diseases usually target our multiple health behaviors simultaneously. And if we want to look at the stages of change for multiple health behaviors, that it actually results in many different response patterns just as shown illustrated by this graph. So in order to tailor the intervention for each stages of change pattern, usually it requires individualized intervention regimen. That is probably why most previous studies that applied the stages change model to multiple behaviors simultaneously often relied on computer-aided programs to facilitate tailoring the intervention for each participant. However, many behavioral intervention strategies are more cost-effective if implemented in a group setting. Furthermore, it has been shown that the emotional support and as well as experience sharing during a group intervention session has been very important for successful behavioral change. Yet, if we want to take, it's not very easy to tailor the intervention in a group intervention setting though. So if you want to individualize intervention strategy, it's not very easy to do that in a group intervention session. Therefore, if we can identify a few subgroups among the participants based on their stages of change for multiple behaviors, it may help design the intervention tailored for a few classes. In other words, the advantages of the stages of change model and the multiple group intervention may be combined. So in order to find a few subgroups among the stages of change for multiple behaviors, I'm going to introduce a statistical method that so-called latent class analysis. The latent class analysis, it's a statistical method that has been used to identify latent subgroups through a set of observed multivariate categorical indicators. It provides a parsimonious and intuitively meaningful summary of cell frequencies in a high-dimensional contingency table. So if we're going back to the stages of change for multiple behavior problem, basically latent class analysis could be the ideal way to identify a few subgroups and reduce the dimension of the high-conditioned tables. If you are familiar with factor analysis, actually latent class analysis, it's analogous to factor analysis for continuous observed variables. So it can be used to parse out measurement errors in some cases. This statistical method has become popular in many different research areas. So I'll just give you a few examples. So in psychology, sometimes if you measure individual for multiple depression symptoms, these symptoms could be categorical, either binary or with several categories. Then latent class analysis can be used to categorize people into different depression categories based on their reports to different symptoms. And also in behavioral science, researchers have used this method to group drinkers based on their drinking behaviors. And lastly, in survey methodologies, it has been used to identify some flawed survey questions. So those are all very interesting applications of latent class analysis. So before I move to an example of latent class analysis, I want to introduce some key concepts of this method. First of all, what are the latent classes mean? They are actually the categories of the latent categorical variable that divide the population into mutually exclusive and exhaustive subgroups. So sometimes we call them latent subgroups, sometimes we call them as latent classes. And one of the most important assumptions of the latent class analysis is the conditional independence assumption, which assumes that conditional on a particular latent class then the observed indicators are independent of each other. And class membership probabilities, which I will denote them by gamma C, are the percentages of observations belonging to each latent classes. If, so for example, if there are three latent classes, then the latent class analysis will report three class membership probabilities, estimates, one for each class, and the three estimates will add up to 100%. And another terminology is so-called conditional item response probabilities, and I will denote them by rho mk given C. So given C means it's a conditional probability. Those are the probabilities of each response for every indicator conditional on latent class membership. So for example, if we know an individual comes from the first latent class, then what's the probability of this individual will choose option one for the first indicator. And from latent class analysis, we will be able to get estimates for their posterior probabilities denotes by as PC given Y, which are the probabilities of individuals being classified in a given class based on the individual's observed responses for the multiple indicators. And each case will have, so basically if a latent class analysis has three latent classes, then each individual will have three different posterior probabilities, indicating that this case is the probability of each case being classified into class one, two, or three. So after all the boring terminologies, I hope I'll give you, I'll make some sense for you by using this graph. So this is an illustrating example of latent class analysis applied to multiple lifestyle behaviors. And in this study, the investigators measured five binary variables and each of them meaning whether the participants adhere to the norm for that particular behavior. So like the first behavior, if a participant adhered to the norm of physical activity, then this participant will report one for that variable and otherwise he will report zero, okay? And similarly for the vegetable variable, it's a binary variable again, the participant will report a one which means yes, if adhered to norm for eating vegetable, and otherwise this person will report a zero for that variable. So basically in here in this study, the observed variables are five binary variables and if you put them together as a contingency table, you basically will get two to the five different response patterns, right? Because each one will have two choices. So that means you will have 32 possibilities in terms of their response, different response pattern. And that is a lot when we want to use those, when we want to analyze all five variables together. And if we want to use them either as a predictor variable or outcome variable. So the investigators of this study, they applied the latent class analysis and they found that there are in total three latent classes among out of the 32 possible response patterns. And the first response, and in this graph, it actually plus the conditional item response probabilities for each indicator conditional on each latent class. So basically, for example, in here, it means that the conditional response probability for the first latent class is about 0.82 to endorse the norm to be adhering to the norm for physical activity. And then similarly, another example, so among the second latent class, the conditional probability for the individuals belonging to this class actually had relatively low probability to be adhering to the norm for the vegetable eating. And then if you look at the first latent class, you will find that it has relatively high conditional response probability for almost all five behaviors. Okay, so all of them are around 0.8. So that means the participants belonging to this latent class, they actually have relatively healthy behaviors for all the five dimensions. The investigator mattered. So that's why this class was called the healthy class, okay, by the investigators called them as the healthy class. And then let's look at the third class. So the third class is almost opposite of the first class. So the conditional probability of this class actually a pretty low for almost all the five behaviors, right? So basically the individuals in this class, they had very low probability to adhere to the norm for all five behaviors. And we call this class as a healthy class. And then there's another class, they actually had relatively high conditional probability for the fit of activity, alcohol, and smoking. But then they had very low probability for adhering to the norm for the vegetable eating and fruit eating. So the authors of this article called them as the pro-nutrition class, okay? So basically this example shows you that, what a conditional response probability means, okay? And also the latent classes are actually the three lines showing the shows here. And it shows that basically, when you have multiple categorical indicators and when you're facing a high dimensional condition table, you can use latent class analysis to effectively reduce the dimension of the data into actually three latent classes, which can be easily used for future data analysis. So just want to give you the model specification of these, of latent class analysis using, so if we assume there are three latent classes based on M observed categorical items, and if we let YI equals be an observed vector for the IC individual. And if we assume that the latent class CI, so if we assume the latent class CI was observed, then the joint probability of YI, the observed response pattern and CI. So in here, YI are like the indicators for all the five physical activities, as I showed in the previous example. So it could be 111, 10 or 10, 111. It depends on how many individuals adhere to the norm for each of the lifestyle behavior. And then CI, if we assume it is observed, it's actually not observed, right? Because we call them latent class. But the first step is that we assume it is observed, then the probability of observing YI, given CI equals to little c, is just a cross product of the conditional item response probabilities times the probability of the latent class, gamma c. So this is actually, if you are familiar with categorical data analysis, this is a pretty typical thing formula to calculate the probabilities of observing your response. And then because we don't know gamma c, we don't know the latent class analysis. So we do not directly observe the latent class analysis membership CI. That is why the marginal probability of the observed item responses pattern for each particular response pattern for each individual, it equals to the summation for each potential latent classes times the latent class membership and then times the cross product of the conditional item response probabilities. So the parameters in the latent class analysis are usually estimated by maximum likelihood using the EM algorithm. And the EM algorithm stands for expectation maximization algorithm. But if you don't know it, I guess I explained it, doesn't make too much difference. But the reason we use EM algorithm is that we actually treat the data on the latent class membership C as missing data. So the model estimation, the standard arrows are actually another byproduct of the EM algorithm. So we can use the first derivative of the likelihood function. And however, it could yield standard arrows of zero for parameter estimates on the boundary. So another way is to take the inverse of the hazing matrix, but the standard arrows cannot be obtained if one or more of the parameters are estimated on the boundary either. So there's a technical difficulty for getting for estimated standard arrows for latent class analysis. And then there are some people proposed robust standard arrows using sandwich type of estimators which rely on both cross product matrices and the hazing matrix. And there's also, people have proposed some basing approaches. So if we can use a mild smoothing prior or using data augmentation algorithm, then the problem with estimating standard arrows could be alleviated sometimes. And for latent class analysis, one of the major step, the first step is to determine how many latent classes there are. Because we don't know, you know, beforehand, you know, if there are two latent classes, three latent classes, or five latent classes. And there are different model selection criteria have been proposed, including the information criteria such as AIC and BIC. And all use some likelihood based, likelihood ratio based statistics such as G square and entropy based measures. And another principle is to avoid solutions with low class counts. And also parsimony and the model interpretation has been very important considerations for this type of, for select the number of latent classes. And then we can use Bayes theorem to compute the posterior probabilities. And individuals may be assigned to the class in which they have the highest posterior probability of membership. And then use for future analysis such ANOVA. Latent class analysis has been extended to incorporate the effects of different covariates. So we call those as latent class regression. Basically, the way to incorporate the covariates into the latent class analysis is through the function of gamma C. So now the latent class membership probabilities, gamma C, becomes a function of your covariates XI. And you can also, you can easily write the rates of class membership related to the covariates using a multinomial logistic regression. So it's like gamma C given XI, the probability of CI equals C given the predictor is linked through a logistic function. And this is very similar to traditional multinomial logistic regression, except that the latent class membership is not directly observed, so that it is latent. All right, so I have introduced the so-called traditional simple latent class analysis so far. But remember the one of the most important assumption of latent class analysis is the conditional independence assumption, which may not be met in many situations. So for example, if you have data collected from a multi-side study, then the data are naturally clustered. And in that case, you cannot really assume the data independent of each other, even your conditional on each latent class. So multi-level latent class analysis has been proposed by different investigators. So like Vermont have presented a framework for estimating multi-level latent class and latent regression models. And then Milton and his colleagues have discussed how to fit general multi-level mixed models, including multi-level latent class analysis. And then recently, Dee and his advisor proposed a multi-level latent class model in which class-specific membership probabilities came from a Dirichlet distribution. And the basic methods have also been proposed recently. The applications of multi-level latent class analysis in different research areas is still rare though. So here's the model of a multi-level latent class analysis. It actually feel familiar with multi-level logistic regression. This is again looks very, you know, probably very familiar to you. It basically, you just link the class membership probabilities to the covariates at different levels, level one and level two, through a random effect logistic regression. And you see it's greater than two than a two-level multi-node logistic regression can be used with C minus one random intercepts are specified. So it's essentially the same as traditional random effects logistic or multi-node logistic regression, except that the categorical outcome is not directly observed. It may, it actually could involve high dimensional integral which can be evaluated by different numerical integration methods. And then the interclass correlation is defined as a proportion of variance of random effects out of the total variance which is defined by this formula. And standard arrows can be estimated by computing the observed information matrix or standard sandwich estimator. So, so far there are three different softwares that can be used to feed multi-level latent class regression, including M plus seven and latent gold as well as SAS PROC LCA. And however in SAS PROC LCA, it actually only uses so-called soldo maximum likelihood and sandwich type of standard arrows that it's appropriate for complex survey data. So it's not exactly the same as the model that I presented previously. So just want to show you some, so far different people have proposed different methods for to model multi-level latent class analysis. But none of them have shown a sort of simulation results to show the impact of, if you don't use multi-level latent class analysis, what kind of bias or in terms of parameter estimates and standard arrows you will get. And so that's why I performed some simulation studies to show whether it is important, how important it is or why it is important to model the multi-level structure of your data. So in the simulations, I simulated either 300 or 30 clusters. And then among those with, among the datasets with 300 clusters, I assume there are 10 subjects in each of the clusters. And then among the 30 clusters, I assume there are 100 subjects per cluster. And I assume there are different interclass correlations from small, as small as 0.05 and up to 0.25. And I assume there are two latent classes, three observed categorical variable items. The reason I assume there are three categorical items is because for the stages of change example, I actually have three different behaviors. So that's why I assume there are three different categorical items. And for each of them, there are five categories. It's also because for each of the stages of change variable, there are five categories from pre-contemplation up to maintenance. And then I assume there's one continuous level one covariates for standard normal distribution with regression parameter beta one equals minus 0.5. And assume there's one dichotomous level two covariates with regression coefficient alpha one equals minus 0.05, minus 0.5. So a sudden data set was simulated for each scenario. So here are the simulation results. Basically, you can see that when we have 30 clusters with 100 individuals in each cluster, the relative bias of the parameter estimates for both level one and level two covariates increase with the increase of the interclass correlation. And then, however, if we use a multi-level latent class analysis, then the bias were relatively minimal. And then similarly, when we look at the coverage rate of the 95% confidence interval, and you can see that if you use simple latent class analysis, if one of the interclass correlation is relatively small, then you don't have much, the coverage probability is still, excuse me. It's still pretty good for the level one covariates, but it's already pretty poor for the level two covariates. And then when your interclass correlation increases to 0.1 or 0.25, then even for the level one covariates, you know, the coverage probability was not very good. So, but on the other hand, the multi-level latent class analysis shows, you know, relatively good coverage probabilities that's near 95 as it's supposed to be as shown in here. And we observed similar, you know, observations for the other scenario when we have 300 clusters and 10 subjects in each cluster. So again, the simple latent class analysis gives bias especially when the interclass correlation are large and the multi-level latent class analysis don't have as much bias and the coverage probability has similar performance. So basically the simulation shows that, you know, if as long as you have a level two covariates in your latent class analysis model, it is very important to use the appropriate analysis method to investigate the relationship of latent class membership and your covariates effects, okay? So now I'm going to show you some real data application results, the data came from the so-called Special Diabetes Program for Indians, Diabetes Prevention Program. It is a translational project that was founded by Congress in 2002 in recognition of the huge diabetes disparities that American Indian and Alaska Native are experiencing. So basically the Congress directed Indian Health Service to implement some evidence-based intervention programs to prevent diabetes and cardiovascular disease among American Indian and Alaska Native communities in the real world settings. The project starts in September 2004. So the diabetes program is one of the two arms of the demonstration projects founded by Congress. And it starts in September 2004, which and it focuses on the primary prevention of diabetes by implementing the diabetes prevention program lifestyle curriculum among individuals at risk for diabetes. The eligibility criteria is basically American Indian and Alaska Native adults with pre-diabetes. So 36 grantee sites from diverse American Indian and Alaska Native communities all over the country participated in this project. And baseline data collection finished in August 2009. About 3,000 participants responded to the baseline questionnaire. So this is just a map of the sites of the SDP IDP program. So, you know, you can see that there are 36 sites all over the country. The red ones, all the diabetes prevention projects are the grantee sites. And then the yellow ones are actually the other arm of this program is the cardiovascular disease prevention program. And the current center is located in Denver, Colorado. So just a quick preview of the design of this SDP IDP. It's actually a fairly simple design because we are using a proven intervention strategy that has been, you know, proved very highly effective in randomized clinical trials previously. So we did not have a control group. So this is a one arm intervention. All the participants, you know, after recruitment they went through a baseline assessment and then all of them started a lifestyle intervention as well as some community-based activities. And then we measured them at four to six months right after the intensive phase of the intervention. So basically after the completion of all the 16 classes of the curriculum and then we measured them annually. So in terms of the measurements I'm going to use for this particular study, we measured for each individual their stages of change for three different behaviors including regular exercise, healthy diet and weight loss. And so each individual participant responded to three questions that will classify them into one of the stages of change, you know, running from pre-contablation up to maintenance. So it was noting that we didn't really measure a preparation stage for the weight loss indicator. So for that variable, we only have four categories instead of five categories. And then the individual level characteristics I'm using in this study including social demographics and the site characteristics including rural sites versus urban sites and some organizational types like IHS versus tribal programs. Sorry, can you say again why you didn't do preparation for weight loss? Yeah, that's the instrument we selected to use. It didn't have the preparation stage. Yeah, exactly. That's a good question actually. That question may come up later as well. So here's the model selection criteria for when I run simple latent class analysis. And if you look at their, you know, AIC, BIS and G-square you will find class number, you know, three classes give to the lowest BIC and relatively low AIC. And you know, there are some other criteria we use that is not shown in here also indicate that three classes it gives you the best fit for your data. It also gives you the, you know, the best in terms of interpretation ability. So basically if you plot the fit statistics you will see that the three classes and four classes are possibilities and three classes are more parsimonious and easier to interpret it. So we chose to conclude with three latent classes. And then the first class is the, in the first class basically it has relatively high conditional response probabilities for both the diet and exercise that, you know, response to the contemplation stage. And also for the, even for the weight class it also has relatively high probability to be classified in the contemplation stage. So that's why we call this latent class as the contemplation class. And then for the preparation, for the next class if you look at the first two behaviors, you know they have very high conditional response probabilities for the preparation stage for both of these behaviors. And then for the third behavior, the weight loss you will find that it actually has zero probability for the preparation stage. And that is because we didn't measure that stage. And then you also find that for the weight loss it has very high probabilities for both the contemplation and the action stage. So the adjacent stage for the preparation stage. So that's why we call this class as a preparation class. And then the last class, the individuals have relatively high item response probabilities for both the action and maintenance stages. So we call this class for all three behaviors. So we call this class as the action maintenance class. And then we just want to see whether our latent class membership makes sense by looking at the relationship of the latent class membership and the behavioral indicators. And as showing here that the wrap up actually are measures of physical activities for aerobic and strength and flexibility. And then the healthy diet school, a healthy diet school and BMI. We found that those who were at action and maintenance stage had most of the physical activity level, the highest physical activity level. And they need most of the healthy diet school. Their healthy diet school is the highest and had the lowest healthy diet school and also had the lowest BMI. So this confirms to our expectation that those who were in action and maintenance stage had the healthiest behaviors. And then when we looked at the latent class membership relationship and with some psychosocial factors, we found that all similarly, those who were at action and maintenance stage had the highest perceived health and coping scale as well as the highest family support and also the physical and mental health related quality of life. So this also confirms that the three latent classes works as we expected. And now since, again. Can I ask a question? Sure. So the latent classes that you're using are derived into the random variables. And so are we taking them to a company in certainty in the class assignment? Yes, actually in these numbers are not because I was assigning each cases based on their highest posterior probability. But then in this one, let me see, the LCA regression at the p-value when we did incorporate the uncertainties of the latent class membership. And we did latent likelihood ratio test to test the significance of the... So how is it incorporated in the circuit and through a prior on the classes? Or is it... It's through the... No, it's actually through the EM algorithm. So basically you assume the latent class membership missing. So when you feed the probabilities, when you feed the models, you estimate the probabilities based on that way. So you're doing it joining me. If you were gonna take the latent class then, though, and model it as a derived variable for something else outside of that frame. Yeah, you can do it that way. That's an easier way. But then the better way, it has been shown that if you take the uncertainty into consideration, it actually gives you more unbiased estimate for the association between the covariates and the latent class membership. Yeah, and it has been shown that, and that's why methods has been developed like latent class analysis regression has been already developed. And then there's another method development trying to use the latent class membership as the covariates of a regression. So basically treating those... So use the latent class membership to predict some distal outcomes. So that is under development as well. So just for easier presentation, I showed you these numbers from the ANOVA analysis, but the p-values confirm the results. All right, so remember our data is, STPI, DP, it's a multi-site study. So it's naturally clustered because the participants coming from the same site probably may have some different characteristics compared to the other clusters. So there's an inherent heterogenearity. That's why I started to look at multi-level latent class model to see if that fits our data better. And these are the model selections I fit and you can see that the model is two random intercepts that the perfect correlated has the best fit statistics in terms of BIC. And also when we have four different latent classes, when there are three random intercepts, the model has very hard time to converge because we have very high dimensional integration problem. And after we fit in the multi-level latent class analysis, we find that actually the interclass correlation are not very huge, which is quite typical for multi-site studies. You know, multi-center clinical trials usually find an interclass correlation around 0.05. So again, here are the three latent classes, the conditional probabilities for three latent classes. They are essentially the same as what we got using the simple latent class analysis. And then here are the results of the latent class regression. For individual level, so you can see that I showed you, I presented both the results from simple latent class analysis and then from multi-level latent class analysis. You can see that for the individual level characteristics, the parameter estimates and standard error estimates are fairly similar, they are not changed so much, but then for the site level characteristics such as rural versus urban, then the parameter estimates was changed quite a bit and plus, you know, the standard errors which was changed substantially as well, leading to quite different conclusions. All right, so just some discussion. Stations of change for multiple health behaviors may be summarized by a latent class model and class-specific interventions may be designed to deliver lifestyle intervention more effectively and efficiently. Latent class analysis allows us to check the validity of the original stages of change measurements. So for example, the algorithm for assessing readiness to change for losing weight might be improved if we add a preparation stage. Now, latent class analysis is effective way for also for data reduction. And so once after applying that, the analysis of multiple state of change variables either use as outcomes or as covariates can be significantly reduced and simplified and streamlined. The conditional independence assumption of traditional latent class analysis may not be met in different scenarios such as multi-site studies. And the multi-level latent class analysis allows us to investigate latent class structure using data with clustered data structure without assuming conditional independence. And ignoring the interclass correlation of observations from the same cluster may lead to inaccurate estimation of the model variances and the effects of the covariates, especially for level two covariates. So here are just some future directions. So we plan to extend the simulation study to more complicated situations. And we also plan to compare the results of different estimation methods, such as a Bayesian estimation using MCMC method. And also latent class analysis with these two outcomes can be used to analyze the relationship of latent class membership with intervention outcomes. So here are just some limitations of the study, due to time limit, I'll skip that. And I would like to thank all my collaborators from the Centers for American Indian Alaskan Native House at UC University of Colorado Denver and also our founding agency, including Indian Health Service and American Diabetes Association, which enabled me to have time to work on the multi-level nature of the data. And as well as all the IHS Tribal and Urban Indian Health programs and the participants who involved in SDP-IDP. And here are just some references. And finally, I would like to thank you all for your attention and time. Any questions? So to talk, I have a question about the standard errors. Well, when I think of clustered observations or dependent observations with that cluster, I think of controlling for that as essentially leading to larger standard errors that if you didn't control for clustering. Yeah, for level two. The information for each individual isn't as great given that there's a tendency. But when I look at their estimates, it looked like the standard error is actually that smaller. Yeah, that's a good question. But I think your understanding for the standard errors are actually mainly for level two covariates. It's not necessarily true for level one covariates. TJU, are you referring to this slide? Yeah, so I see, right. And so I want that site level to move to the last row because it seems that your method which is a better correction method is actually the standard error. The last row actually, it increased the standard error estimates for the site level characteristics. It's not, if I'm, but you may observe some different directions for the individual level characteristics and that is possible for level one covariates. So the comparison isn't, so your model is the A and for C. It's actually a multi-level versus the standard level. Which standard error should be the standard level? This is a single level standard error. So this panel, and then these are the standard errors coming from the multi-level class analysis. That's better than the previous one is. Which standard error? What do we mean? Are you talking about? I'm comparing the 0.18 to 0.04. But I shouldn't be doing that, right? So what should I do? Oh, yeah. This is actually the standard errors for comparing preparation class to the contemplation class. And then this is action class to the contemplation class. So both of these are from single level written class analysis. Yeah, yeah, yeah, exactly. That's the correct direction. So it depends on whether the covariate is varying within the cluster or varying across the cluster. Yeah, for level one, it could go down. The direction of bias is not a table. And from one level, the direction of bias is part of the table. So do you have an explanation for that? Let me move to this slide, and then can you repeat your question? Sure. So if you have a simulation for that. Yeah. When you look at the simple LCA, so we think that the bias is narrative. Most of them are narrative, right? Oh, yeah. And for the model level, most of them are politics. So you have an explanation for that. That's a good question. No, I don't, I just, I really about this directly from the simulations. It could be because I assume the relationship between each of the covariates and the written class membership would narrative. So I don't know whether that has anything to do with that. Yeah. It's a good observation. Any other questions? Okay. Thank you very much. We usually have a...