 Marker variables are commonly recommended as a solution for method variance problems, and they're often applied in research practice. In this video, I will explain the concept of a marker variable, some features that marker variables should have, and also give an overview of two statistical approaches for working with marker variables. The idea for marker variable is commonly attributed to this article by Linder and Whitney in Journal of Applied Psychology. They started from the assumption that there is a single source of method variance that affects all items equally, or at least to the same proportions. But there is one source of method variance, and these constructs are another source of variance. Now, if we have two constructs that are uncorrelated, then the only reason why their measures would be correlated is because of the shared source of method variance. If we can find a construct that we can actually measure that is unrelated to the key construct in our study, then we would call the measures of that construct marker variables. So we need to have a theoretically unrelated construct. For example, something that people have been using is whether the person likes jazz music or not. When you measure, for example, firm performance and firm innovativeness using a five-point agreement scale, then measuring whether the person likes jazz music on the same scale format could be used as a marker, assuming, of course, that that marker is affected by the same kinds of biases than the actual variables. But whether a person likes jazz music or not should not in any way be related to the performance or innovativeness of the company that the person responds for. So this is the idea of a marker variable. You find a theoretical unrelated variable and then there are a few different ways of estimating the effect of method variance using the marker. The approach that Lindel and Whitney recommend is taking the smallest correlation between the marker and all the other items or if markers that are prior designed are not available, then take the smallest correlation of all the study variables and use that as a proxy for method variance. This is called the ad hoc marker correlation approach. Marker variables have some merits. So this is actually a technique that can be effective, but it has some limitations. And marker variables are commonly misapplied. One of the most common ways of misapplying this is that you choose a marker that is not affected by the same source of bias as the items. For example, if you ask a person to rate things on a five point scale and then you ask them to report their age and you use the age as a marker, it's hardly true that how a person responds to a question about their age would be affected by the same kind of biases than how they respond to a rating scale about company performance or innovatingness. There are also misapplications that relate to statistical techniques and some of the techniques recommended in literature are actually a bit questionable. I will point out some issues here in this video that the existing literature has not pointed out this far. Ad hoc markers generally don't work. So if you try to assess the effect of a method variance just by looking at the magnitude of the correlations, you'll run into the same problem that you have with those harm and single factor tests and other simply correlational techniques. It is impossible to say if the high correlation between items is because of a method or because the items measure constructs that are highly correlated. Such models are really identifiable. I'll talk about the identification of those models a bit in another video, but generally ad hoc marker techniques don't really work and should not be used. So a prior chosen marker where you can have a theoretical unrelated construct that is measured may work choosing the smallest correlation among the interesting constructs or interesting measures does not really work. There are a couple of ways of using markers and the confirmed factor analysis way that I'll explain in this video is currently the preferred way over another way, which is called the correlation technique. Let's take a look at markers in more detail. The idea of a marker variable is explained really well in this article by Simmering and co-authors. They are explained that the marker should have two key properties. One I already mentioned should be unrelated to any of the study variables. So if you have innovativeness measures and you think that they are suspect to social desirability bias, then you need to have a marker that is unrelated to innovativeness. The construct should be unrelated to innovativeness. For example, whether a person likes jazz music, whether they think the weather is nice outside, something like that. Then another important and often overlooked feature is that the marker should have the same sources of bias as the interesting variables. And this of course assumes, if you have a one marker, it of course assumes that all indicators that you have are affected by the same kinds of biases. For example, if you think that there is a leniency bias and social desirability bias that affect your items, then your marker should be influenced by those same biases. So if people are more lenient when they self-evaluate, then the marker item should be something that people also answer more leniently to, the same for social desirability. This is an important and often overlooked feature of markers. So two important things, theoretical and related variables and share the same source of bias. This article by Spector also talks about marker variables and provides a general overview workflow for how to deal with method variance issues. And they advocate modeling different sources of method variance, for example, social desirability and extreme response style and other things. And if you have items that are affected by multiple sources of bias, then you need to have multiple different markers. They also make a good point of dividing constructs or things that we measure into these five different classes. And generally, if most of your constructs are, that of interest are, for example, behavior constructs, then your marker should be behavioral as well because behavioral constructs are affected by different kinds of biases than, for example, factual measures. If you have factual measures, if you ask persons height or persons age, there is probably very little bias there. Maybe a person who is 36 might say that they belong to the 30 to 35 category of age because they would like to be a bit younger. But generally, the biases in these factual measures are pretty small. And they are certainly different than, for example, these evaluative measures. For example, self-lineacy would not probably affect factual measures, but would affect self-evaluations. So you need to think about what is the kind of variables that I have, what are the sources of bias, and then what kind of marker could I use that belongs to the same class of constructs and also the items would share the same source of bias. There are also variables that are specifically designed or developed to be markers. And this is a nice example. There is a construct called blue attitude which measures whether the person likes the color blue or not. And it's very difficult to come up with a reason why that would correlate with anything of interest. But nevertheless, measures of blue attitude can be used to gauge, for example, extreme response style and other sources of method bias. So this is one way so you can just develop an arbitrary construct and then develop measures for it and use that, and this has been used in multiple different studies. Then there is also attitude toward neutral objects, for example, how much you like public transportation. One problem with using these kind of markers is that if you are, for example, measuring things from CEOs, you are using CEOs as informants, and if you start asking questions like whether you like color blue or whether you like jazz music or whether it's nice weather outside, that kind of things in the middle of the survey that asks about company strategy or company innovations or performance, that's gonna be pretty weird for the informant. And they may switch to a different responding style with those weird items. So one practical thing when you choose your marker should be that even if it would be ideal to use a marker that is unconventional for your survey, you need to take care that the informant does not feel weird when answering the question. Here's another example of perhaps not so good practice, but this is fairly common practice in using marker variables. So they are took by Tivana in strategy management journal, did a cross-sectional survey. They measure things about technology and things about outcomes. And all the interesting constructs were measured. They were rating scales from one to five, strongly disagree, they strongly agree. And these are, most questions are something that would be affected by social disability bias. So you want to say that, for example, they are, relationships are stable and well understood in their events. So do you want to agree on these items? But then what are the markers? So they applied markers and this is a pretty good explanation because there's actually quite a lot of transparency here and they explained the principles. But when we look at the markers, the markers that they have is the existence of controlling firm operations in South America, the counter vertical industry segments in which the controlling firm operates and whether the project solved was with Microsoft Windows. So how would these factual statements be affected by the same sources of bias than these statements where the person evaluates how the company is doing? It's very difficult to see how these items would be affected by the same, for example, social disability bias. And indeed this study found no evidence for method variance. And well, that's pretty obvious because these markers probably don't capture any of the sources of bias that may affect these items. This does not mean that the study has a method variance problem, but this evidence that they presented is not valid evidence for not being, for there not being a problem. These kind of markers are referred to as a shoe size marker variables in the literature. The term was coined by Potsakov and Mackenzie and co-authors and they explained that shoe size persons height are the kinds of things that are factual or demographic variables probably don't work well as markers when you are measuring attitudes. The same is explained perhaps a bit better in this article by Williams and co-authors. They also use the term shoe size markers. So it's critically important that your markers are affected by the same sources of bias. There's also evidence that not even all the markers that we measure on the agreement scale like one to five or one to seven are affected the same by these different sources of bias. There's a study about markers by Simmering that anyone who uses marker variables should read and understand really well before proceeding with your study and they study different markers. The markers that they have, they have these six markers here, they have the blue attitude, then they have objective or factual marker, they have tenure, they have evaluating markers, they have attitude markers and that kind of things. And they found that when they measured different sources of method variance here, for example, self-deception, over-claiming, extreme response, either negative or positive, these markers, some of them are insensitive to these method bias sources and those that are sensitive to the method by variance sources are sensitive to different sources. So it's important that you match your interesting variables, how they are affected by the sources of method variance. And you choose your markers so that they're affected the same way because not all markers are equal. How do you know whether, for example, your markers are affected by self-deception bias? Well, one is just to think through the item. Would it be something that people like to respond positively or negatively? Another thing is that some studies that have actually applied the marker before could have also measured something like this, a study measures these different sources of method variance. So there's some evidence, not a lot, but some evidence on how markers work and you can use that evidence to make decisions on which markers to add up to your study. Now, let's move on to statistical techniques on how these markers are applied. There are two main techniques. The Linda Law Whitney article originally presented this correlation of marker technique. And their technique basically has two variants. You can either choose a correlation based on a prior chosen marker. So you choose a correlation between a marker indicator and your key variables, or you choose an ad hoc correlation, which is the smallest correlation among all your study variables. So if you use the ad hoc technique, then you wouldn't have marker variables that are designed in the study, but you just use a correlation between the interesting constructs as a marker. And then you parcel the marker correlation from the correlation matrix. The parceling here basically means that you subtract the correlation from all correlations in the correlation matrix. It doesn't exactly work that, but that's the idea. So the actual math is slightly more complicated. But the idea is that if there's a small level of correlation in the data, then that is due to the method, then you take that correlation away from all study correlations and then you assume that those correlations that are left are purified from any source of method values. So this is the correlation of marker technique. Then there are confirmed factor analysis marker techniques. And these confirmed factor analysis marker techniques basically involve fitting the normal factor analysis model and including the marker items in that model. Larry Williams talks about these models and his 2010 article presents one of the most commonly used workflows for how to do this kind of analysis. So let's take a look at the problems of these items. There are marker correlation approach has two main problems. One is sampling error. And this is acknowledged in the original article as well, but they don't play its importance. So the problem basically is that if you choose the smallest correlation, then you are going to be choosing a correlation that is affected by sampling error. If your sample size is small, then one of the correlations or one or more of the correlations will be small by chance only. So here we have correlations of 10 variables from a sample size of 100 and the population correlation is 0.25. The smallest correlation is 0.06. So is the 0.06 representative of the overall correlation level between these variables? The answer is no. It's negatively biased because we choose the smallest correlation and this bias increases when the number of items increases and it also increases in small samples. So we measure the smallest correlation 0.06 and the population correlation is 0.25. So choosing the smallest correlation will indicate that there is no common source of variation in these items where in fact there is. In practice, I've never seen an article that applied this ad hoc correlational market technique and did not conclude that the smallest correlation is close to zero. In practice, there's always one correlation or more correlations that are close to zero and unless your sample size is very large and your theory is very strong and then you can basically just conclude that the smallest correlation is small, therefore no problem. So this is kind of like a get away from jail free card. It doesn't really work that well. So this technique using ad hoc correlation can seriously underestimate the magnitude of the problem. Then there's another problem. I mentioned this also in another video but are parceling out a correlation is basically equivalent to fitting a model where you have one method factor that loads on all the items equally. And then when you parcel out this method factor you're basically are modeling these, taking the residuals, what is left after the method and then assuming that that is the variance of interest. Well, there are, the problem with this is that the method implies a constant correlation. But we just saw in the table from Simmering that not all sources of method affect all items equally. So this is an unrealistic model and if you fit an unrealistic and mis-specified model to the data, then the results are generally not trustworthy. There is also the implied covariance or the implied small correlations basically the variance of the method factor. And now the question is that why would we take that variance of the method factor based on the smallest correlation instead of simply estimating it using this model? I'm not saying that this model is useful even if it was estimated but it seems that it is even less useful when you just take a correlation instead of estimating the model fully. At least when you estimate this model and when your software tells you that the model is not identified you know that you have a problem. If you take a smallest correlation and then fix the method factor variance to the smallest correlation then your software will not know that there is an identification problem. Finally, this is equivalent to estimating a factor model of residuals. So it is basically equivalent to running first the factor analysis, fixing all the loadings to be the same and then taking residual covariances and then estimating these factors on those covariances. The problem again is that this method factor also assumes variance or covariance between the constructs so it cannot really differentiate between construct related variance and method variance that way. A lot more defensible approach is to use the actual marker variables and use them in a confronted factor analysis model and this is what the literature recommends. And this is from the Williams paper, 2010 and he shows that the way to use marker variables is that you have these constructs that are correlated and these are the interesting constructs or the factors that represent the interesting constructs and these are the measures of the interesting constructs. And then you have the marker measures and you have a factor for the markers and that factor also influences these interesting measures. And then Williams goes and explains a series of a sequence of modeling of different kinds of nested models that you can do to test different assumptions about the method variance. They are using, in an empirical example, benefits administration. They're using data as a marker and that has been used in a couple of studies by Williams. But now, if we think about that these items measure benefits administration, what does the benefits factor or the marker variable factor here actually present? Is it the benefits administration construct or is it the source of method variance? We don't really know. This is, if we say that these items are correlated only because they share one source of variation, then that one source of variation would be a combination of the benefits administration construct and the method. And you're basically confounding the method variance with the benefits administration construct if you do it this way. And that is something that literature has not really noticed yet. So a much more defensible approach would be to model it this way. So like you're saying that these items that you have here are affected by constructs and they're also affected by the measurement approach. And these items here should be also affected by the construct, which is your marker construct and the method variance source. So this is a more accurate presentation of the data and it also solves the issue that you commonly have that these marker items load very highly on the marker construct and the marker construct does not really affect or marker factor doesn't really affect the other items. So this gives you a more accurate representation of the source of method variance and if there really is one source of method variance then this kind of model should be able to estimate that consistently. And as long as you have a large sample size then the estimate should be trustworthy. Now, there is another issue if you add this second factor here. Now, this is a full by factor model. So you have every indicator loading on two different factors. So if you have a by factor model where the general factor loads on some indicators that don't have a minor factor then that model is generally identified. However, when you have a full by factor model then identification becomes problematic because of these correlations. I'll talk about the identification of these kind of models in another video but at this point it's useful to understand that if you add this secondary factor which makes a lot of sense to do and which the existing literature has not really figured out yet then you run into an issue question of whether this is identified or not. It is identified in some scenarios. It's not identified in other scenarios so it's important to understand. So conclusions on marker variables. Marker variables are potentially useful. There is a lot of bad practice for example, user markers using the correlational technique instead of the converter factor analysis technique not considering whether the marker is influenced by the same source of bias than the key variables that you're studying and so on. But more thoughtful choice of markers if you actually go through the constructs that you're studying and the items that you have for those constructs and then make a list of potential sources of method variants that could affect the items and then choose your markers that in a way that they're affected by those same sources of method variants. Then markers can work. This article by Simmering and then another article by Spectre in 2019 that I cited earlier in this presentation talk about the choice of marker variables in great detail. So if you choose your markers in a way that they actually are affected by the same source of bias then the markers are potentially useful. Also, if you use the converter factor analysis technique and the full by factor model and you can ensure that that is identified then your results are potentially trust for you even if you do a cross-sectional study. However, this is easier said than done and I'll talk about the modeling of method factors in another video.