 Thank you very much. And Imka is also here, so she can help spread the burden or share the burden of interesting questions. Well, it's the light to be to be here in Toulouse without Castle A. So I want to talk about this paper about framework for detection measurement welfare analysis of platform bias. And again, with Imka. So everybody in this, you know, the seminar knows platforms and regulators, it's all on our minds are all kinds of headlines. And in particular, the, we now have laws either being contemplated or put into effect that ban self-preferencing. There are a lot of other concerns about platforms, but, you know, today we want to be focusing on this question of self-preferencing that I think a lot of us have been interested in as well. Now, there's a, you know, there's a, excuse me, there's a sense in which regulatory action is way out ahead of research. And I think there's an urgent need to, to, to, you know, to detect and measure the welfare consequences of self-preferencing. But the thing is identifying unwarranted self-preferencing is not at all straightforward. I mean, it sounds kind of straightforward, but I think in some sense it isn't. So the basic questions that we want to address or try to address in this paper, first of all, what is self-preferencing? Secondly, how might we detect it? And then third, what is its welfare cost? Or at least what I should say is how might we go about measuring its welfare cost? I want to be clear that this paper, isn't it so much about the phenomena as it is about how, how to study the phenomena? Because the, as you'll see as we go, the data we have are all quite imperfect. But there's going to be a silver lining, so don't, don't tune out just yet. All right, so the generic setup that I think the self-preferencing question is typically about is some kind of ordered listing of products. Like search result rankings are maybe the most common kind of example, but you can also think of just the display of products on product pages. And users choose among some ranked list of products. And the platform in the background is choosing the ranks to serve some objective that might involve the welfare of consumers, the welfare of sellers, or the welfare of the platform itself. Now self-preferencing, I'll give a vague definition now, but we'll get more specific as we go. Self-preferencing arises if the platform ranks its own products higher, meaning better, it's a lower number, but higher in the ranking, then would maximize some combination of seller and consumer surplus. So we're going to think about self-preferencing as deviating some sort of frontier involving the interests of buyers and sellers. A roadmap for the 40-minute talk, maybe 35 left. First, we'll give a simple theoretical framework. And then we'll give, this will give us some theory-based ways to think about bias detection methods. And we'll talk about two approaches that are, you know, being used, conditioning on observables approaches versus what we call outcome-based approaches. And we'll show with Monte Carlo evidence the advantages of the outcome-based approach. But to be clear, it's not that one is good and one is bad. It really depends on what data you can have. What we're trying to talk about here is not so much to, you know, to criticize, but rather to point out what are the advantages if you had different kinds of data and you could do different kinds of things. Then we'll go to some actual empirical data from Amazon and Spotify and Expedia to do some empirical comparisons showing how you can get conflicting results from these different approaches. And then finally, we'll implement the framework directly estimating simple versions of the structural model that gives rise to both estimates of the rank bias as well as the welfare effects of whatever bias we detect. And we will see meaningful differences across the settings that we study, which, again, is not so much about whether there's bias in the world, but rather that the methods, I think, can reveal some interesting things. Alright, so the model. Two parts to the model. Consumers are choosing among ranked products and better we're going to build in the idea that's both plausible and evidence-based that better rankings give rise to higher purchase probabilities. The platform in the background is choosing if they're end products to rank, it's choosing among basically end factorial possible rank orderings in what it chooses, how it chooses to depict the products. And in so doing, the platform is deciding, first of all, how to balance the interests of consumers versus sellers, and secondly, how much to advance its own interest at the expense of the consumers and the sellers. That's how we're going to think about a division of the problem. Now without self-preferencing, we're going to think about search rankings, for example, leading to a welfare frontier between maximal consumer surplus and maximal producer surplus. I think where that's the producers here are the third party sellers, of course, when there's also, you know, Amazon selling its own products, it's a producer and a platform, but whatever. With platform bias, the rankings are going to give rise to a departure from that frontier. That's the idea. Now, before I get into the implementation, let's just say what does one need an implementation and what we need from a demand model is a way to map product characteristics, prices and platform chosen ranks, kind of as primitives. Into the quantity sold of each product, therefore the revenue for each product, and the total revenue from the choice set, as well as the consumer surplus from the choice set. There are a lot of ways one might do this. You know, again, in principle, especially if one had the right data, one might think about search models, one might think about limited information choice models. So we're going to illustrate this partly for simplicity of illustration and partly just because our data aren't maybe what we wish they were, we're going to go forward, you know, go forth with pretty straightforward illustrations via logit and nested logit. But again, you know, put in your favorite choice model here, because our idea isn't about the particular demand model, we didn't invent that. Our idea, rather, is how to define self-preferencing, you know, with this welfare frontier idea. Okay, so on the consumer side, consumer I is going to choose among now I just we called it J, we called it and a minute ago, sorry about that, among J rank products and consumer I has this utility function. Now what we've done here is we've separated out this delta J zero, which we're going to call rank independent quality of the products from the part that's causally determined by the rank location of the product. Okay, so that's that's the causal rank effect. And so mean utility of a product is again you bar J is delta J zero plus the gamma times r J, but this rank independent mean utility is delta J zero. Now I want to be clear so if you think about rankings, think about a search ranking or something. There are two different ways in which the rank is related to the product quality. First, there's a potentially causal impact of the rank on purchase like as demonstrated in Ursus nice paper with that randomization. But there's also another aspect here which is that the platform is going to rank better and unobservably better products higher as well. So big note here in a box. Delta J is also related to our J beyond the causal effect, because the platform ranks better products higher, but we do want to isolate out that causal part because that's the part which by re ranking the platform could cause you know different things to be sold and different degrees of consumer surplus to be experienced. Okay, so outcomes depend on the ranking we'll use capital R just to describe the whole ranking of the products. And again, in logit land. So SJ is the is the choice probability for product J and it's got this familiar form. I think all we're doing here with the yellow stuff is just to bring out the part that's induced the causal effect induced by the rank choice. And the gross seller surplus of cross products going to be P my not now let me also make a simplification that I should make clear right now. We're going to go forward pretending and it's also true to be usually for our context that price and variable profit are the same thing. So we'll talk about revenue and variable profit interchangeably, but let's realize or remember then some context that won't be true. And so we've done a little thinking about how to deal with that problem, but it raises some different issues, but anyway, I'm going to start getting sloppy. Instead of writing P minus MC. I'll start to be talking about revenue. In any event producer surplus looks like this in a logic and consumers surplus looks like this. But the point is, all of these things are manipulable by the choice of the ranking. Via the e to the, you know, gamma R because, you know, gamma is negative and so the worst is the worst is your ranking the lower is your, your purchase probability and so forth. So the platforms ranking choice again is a big combinator problem. Now we realize that this is related to some things people have been thinking about about, you know, optimal rankings serving interests of platforms and so forth. Rankings depend on the platforms disposition toward consumers versus sellers as well as bias. And then a couple of extremes and what we're going to call the welfare frontier maximizing CS. Now that one's kind of straightforward, because after all this e to the gamma term, it just is a is a bigger, bigger markdown, right, the worst you rank things. So if you wanted to maximize that some and therefore maximize CS you just put the best in the sense of highest delta J zero products at the highest ranks, and you would just order by delta J zero. That one's kind of straightforward maximizing consumer surplus arises from simply ranking the products by delta J zero. Remember delta J zero has in it. The effects of the product attributes as well as the price already like the minus alpha P in some sense in the usual model has already been subtracted out. The other maximizing PS or revenue that's a little harder, but we're going to propose that you could just rank by, let me just use a shorthand and say PJ times e to the delta J. So as it's going to turn out, that's, that's not as straightforward as it sounds, but it turns out that it works really, really well. The reason it's not as straightforward as it sounds is because, well, this, this, this ranking by, well, maybe a formula help to describe it, but but rather than argue against myself, let me just say we've done lots of numerical experiments and this simple approximation works really well. Okay. So the welfare frontier ranking according to PJ times e to the delta J zero is what's going to maximize revenue or producers surplus and ranking according to e to the delta J zero maximizes CS. And so this is just the downward sloping portion of this welfare frontier. So what we're going to do is define essentially the supply side of this problem via this this this function or this this this relationship so I J star is going to equal Kappa one times the natural log of PJ plus Kappa two times delta J zero. So if you think about that, this is essentially, we're essentially describing some family of functions that involve just ranking according to consumer surplus versus ranking according to something like P times Q. So if you think about values of Kappa, if Kappa one equals Kappa two, this is equivalent to ranking by something that's monotonically, you know, monotonically, or monotonic with revenue P times Q in some sense. And if Kappa one equals zero and Kappa two is positive, it's just ranking according to consumer surplus all that matters for the ranking is again the quality of the products. So, the way we're going to think about bias then is to add in another term to this supply function. There's a box here in this this plus I guess that's a side times delta DJ DJ is just an indicator for the product, the product being, let's say a platform product, or in other contexts it might be a platform preferred product you know but whatever it's some kind of product about which we suspect bias. So we're going to allow for for this this additional term in the supply function that takes you off the frontier. So if size not equal to zero there's bias and it's going to change the ranking, and it's going to deliver a solution to this problem that's interior to the frontier. Okay, so something like this is just hypothetical. So something interior, and then you can think about the welfare cost being the some measure of the distance between that term and the frontier. Okay, so let's let's try to implement or use this idea, go from theory of bias tests. So the supply function and bias detection that we're going to call this the conditioning on observables approach. Now, so I, you know again equals Kappa one times natural log of price plus Kappa two times delta J zero plus side times D and that there's some error term. Now, if one were to imagine that this, this supply function were both cardinal and linear, hence the joke here, but cardinal and linear. Then one could just write the following, it would be that is our supply function would be equivalent to just the rank is equal to some Kappa prime times L and PJ some Kappa two times delta J zero plus some psi prime times D. One could regress ranks on these terms and the coefficient on DJ would well reflect bias. So this is the conditioning on observables approach through the lens of our setup. I think, in reality, the delta J zero is hard to observe because the delta J zero is it's easy to observe if you have quantity it's like a mean utility that you get out of a demand model but if you don't observe quantity, then it's kind of hard. So, you know what what people tend to do in reality for understandable reasons is they'll run a regression of RJ zero or RJ excuse me on some controls X, and maybe they'll put the price in there or not. That's not super important. I don't think and then an indicator for the for platform products. And so, so I provides a measure of bias effects controls for all the effects, or at least that's actually too strong a statement but if it controls and doesn't leave out uncorrelated stuff. But the thing is the side could also reflect on observe platform brand characteristics. You know, so for example, Amazon basics batteries might be desirable more desirable than their observable characteristics reflect or less desirable whatever. Yeah, there's some Amazon basics products. Okay. Now the other approach. So, so again, the just a, you know, to depart from the conditioning on observable approach. It's an utterly sensible thing to do if, as is quite typical, you can observe ranks you can observe characteristics. And so it's I think it's very much of interest to know, you know, whether there's differential ranks of Amazon, for example, products conditional on stuff, but there's a lingering worry that there are some observables that that are also correlated with the with the platform dummy. So the other approach, the outcome based approach, you know, is to is to basically ask whether conditional on the ranks that platforms assigned whether products sell differently and it turns out I think we can interpret that approach through this lens as well. So in the platform product, it's going to have well what's shown here Kappa one times Alan pj plus capital to and so forth, whereas a non platform product is going to have the same thing without the bias term. So in the absence of bias of course size equal to zero, they'd have the same quality expected quality, if size positive than the platform product at the same rank would be would be a worse product. If, if gamma that is if the causal effect of the rank is the same for platform and non platform products, then this gives rise to a kind of a very simple outcome base test for bias. Just ask the quantity essentially pursuant to the ranking, the log quantity stick in rank fix effects so conditional on rank and again you could control for price to be fully consistent with this framework, although I don't think it matters terribly conditional on rank to the platform and non product platform products sell differently kind of pursuant to the impact of their ranking on their on their sales. Now, the outcome based intuition, you know with revenue maximization the platform would assign ranks based on expected quality and better quality, better products would get better ranks, and the effective rank on sales is multiplicative. If there's no bias that you'd have the same expected sales for platform and non platform products conditional on rank. If they're self referencing you'd have a lower effective rank on sales for platform versus non platform products in some ways this slide is more intuitive than the last. All right, so implementation and data needs and this is not you know for all the kinds of approaches we're thinking about. One needs rankings and platform identifiers and then for the conditioning on observables approach one needs like characteristics of the products that you know the ones that belong in there. For the outcome based approach one also needs to observe the outcomes that the ranks in part cause and in part are correlated with like quantities, quantity sold to do the welfare analysis. So everything above plus an ability to get causal rank estimates because in a counterfactual you want to change ranks and say something sensible about what would happen with counterfactual ranks. Yes, so it's time for horse race. Now, so one thing we're going to do here is a Monte Carlo simulation with for the following idea. So let's suppose that the expected sales of well to try to try to see when the conditioning on observables approach works versus when the outcome based approach works or really when one when one is vulnerable. So expected sales depend on beta X plus tau Z and on and on the rank assigned. Now let's suppose Z is observed by the platform but not by the researcher. So these depend on X and Z as well as the platform indicator, but here's the problem. The platform indicators potentially correlated with the unobserved thing. So we can, we can set up a little Monte Carlo in which we, we have all these features and we can run the regressions the conditioning on observables regressions and the outcome based progressions. So what we do is we simulate data for a range of biases degrees of bias and a range of correlations between the unobserved product characteristics unobserved to the researcher observed to the platform and the platform indicator itself. And, you know what we end up with the sort of a pretty picture. There's a good color on this picture and things that aren't yellow or less good. So what this is saying is that the conditioning on observable approaches are on the left hand side, and the purple purple what those are our values to detect bias. Yeah, these are essentially the test getting it to test getting it wrong. This is, yeah, this is all conditioning on observable, excuse me, I misspoke about one thing. The whole thing is conditioning on observables. The lower right is one where there isn't anything unobserved that is the, we assume that the Z can be controlled for and observe the full information set up. So if, if we do it with the outcome based approach, we see that we do as well essentially is observing everything, even though we don't observe everything so the right hand side picture here which is yellow is showing us getting us results as good, essentially as if we could observe everything, even though we can't observe everything. Okay, so again I mean I don't know if we needed a Monte Carlo for that the point is unobservables could threaten one kind of test if you could in addition to observing ranks also observe some outcome that the rank is meant to affect, then there's another approach available to you. So some real world illustrations and we have illustrative data but all these data have have weaknesses to go with whatever advantages they have. So if we have the Amazon Kindle daily deals, what is that what that is is every day Amazon promotes 50 books, and they rank them and they post this ranking on a page they also send out emails to subscribers. Now, it's an interesting context in part because Amazon is a big publisher. And so self-preferencing is a real potential concern and interesting concern here about 20% of the books that they promote are published through Amazon Direct or Amazon publishing. So again, there's the possibility of self-preferencing. We also have the Expedia hotel searches that many people have used these are old data now from 2013. We have like 400,000 searches and 8 million listings. Now what's interesting about these data is a bunch of the searches are randomized. Now there's no self-preferencing in this context because Expedia is not a hotel company that said there, there might be bias with respect to chains. I don't think that's not so much a credible concern as an illustrative as an illustrative idea. Then we have Spotify New Music Friday data which I've analyzed in the past with Luis Aguirre and another person, another wall folder I guess. So there we look at 20 songs by country ranked each week and there are whatever 20,000 listings and of these about 6600 eventually appear in the top 200 streaming songs. And the question there is, is there possible bias with respect to the major record labels who are partial owners of Spotify. So again, it's not quite a self-preferencing context, but it is one where it's a platform that's doing stuff and you might worry about its bias. Okay, so I'm going to talk at least about part of this. The first column, so this is the Amazon regressions. The first column is just a regression of log rank on in this case the preferred indicator is, is it an Amazon product? And there are a bunch of controls that we're not reporting in this table. But after accounting for stuff, Amazon products have lower that is better ranks. So that looks like self-preferencing. The second column is the is the outcome based attack outcome based approach. So what this is is a regression of log expose quantity on rank indicators, the price and I forget if we have anything else in there. This one says this was negative as well so they both indicate self-preferencing they both indicates that the direction of, you know, of the measures are the same for this test for both Amazon tests are the same for this context. I mean that said that the sizes of the estimated the sizes are rather different, but the directions are the same. Okay, I for time I'm not going to talk about those last two columns although I suggested that I just gave you some suggestion from them. Here's the Expedia context. So here ranks sorry chains in the first column. This is again a regression of rank on whether your chain and some other stuff ranks get worse. Sorry chains get worse that is higher but worse ranks. And the second column is the outcome based approach. This again indicates, if anything, anti chain bias. Both find that chain hotels are ranked too low, although again the magnitudes when you convert the magnitude of the of the outcome based test into a rank effect which for which you need to make a parametric assumption about the effective rank, you get pretty different sizes but the same direction. Over at Spotify. Now the first column shows that this is major major label music is being preferred according to the conditioning on observables tests again there's a long list of observables with funny names like dance ability, because these are songs, dance ability and liveness and whatever, but a lot of Conditional conditional on those the ranks are better for major label music in the outcome based approach by contrast. The, the, there's bias that there appears to be bias against major label music because it streams better conditional on the rank to which it is assigned so that we're getting different not just different magnitudes but different different signs from these two tests in this context. So the bottom line is that the field data, at least provides some evidence for the Monte Carlo results and concerns about the context in which the conditioning that there are contexts in which conditioning on observables might give you the wrong answer. Okay, so structural but again I mean, there's no point in criticizing something, unless you have an alternative and you only have an alternative if you have quantity data and so forth so we want to be clear that it's not like we think that's a hopeless approach in fact, I suspect you'll see us doing more stuff with that as I think we just want to understand when it's when it's best applied and when you know when it's best justified and whatnot. All right, so the structural approach. So let's start with the. So here we'll just do Amazon and Expedia because we don't have price data that the Spotify context doesn't have a product price there's a subscription price. So it really is not so amenable to this to this stuff if we want to do usual sorts of welfare analysis, but we'll do Amazon and Expedia. And at Amazon, we're going to estimate a plain logic, almost without apology. And so X is going to contain an Amazon dummy and some pre promotion sales. And so the estimated values minus, you know, causal gamma are going to give us our Delta J zero hat. On the supply side what we're going to do is we're going to look at the relationship between the ranks that the platform assigns to the products, the prices and then these these Delta J zero hats. And then we'll put in the platform indicator as well. Okay, and we're going to think about, you know, rank ordered logit as the way to do this because we want to be, you know, ordinal not cardinal and so forth, although it's going to turn out that doesn't terribly much matter. So first of all, the Amazon estimates. So if we just run this run this regression and do this this logic. We get this rank effect of minus point 405, but we also want to get the what we would think of as a plausibly causal gamma. So for that we estimate a different regression that has product fixed effects and we make use of the fact that the same book gets promoted on different days at different ranks. So we take our product fixed effect. And when we do that we get the causal part of the rank effect as shown here is point 335. So I guess that should be a negative sign. So we distinguish between this overall relationship with our and the causal part that we want to use in the counterfactuals. So the supply side, we're going to do three things first column two is not the rank order loaded. It's just the linear regression of the rank on whether it's a platform product, the log price and the rank independent, excuse me, mean utility. It's intuitive just because, well, you can see the sign. So if it's a platform product, the rank is is better, lower, lower means better. That's also confusing about ranks is worse means higher but whatever. So the platform products get better ranks, the higher prices give you better ranks, and higher quality gives you better ranks now that might seem counterintuitive but remember, higher quality giving you better ranks that makes sense consumers like that and it drives demand higher prices giving you better ranks reflects the fact, the idea that the platform likes revenue. Okay, so so it's not it's not weird it's a it's only weird if the post office ran the platform. The third column is the rank order loaded and it's just, it's, we use the normalization so things are positive instead of negative but it's the same kind of results. The other thing maybe to note just as the magnitude of the estimates. The platform product in the third column has a coefficient of 12. Where's the other co I mean just think about how that's kind of big in some sense compared to the other two coefficients, and that'll maybe be more meaningful when you look at an Expedia where everything's going to be infinitesimal, or that is the bias looking stuff's going to be kind of an infinitesimal. So for an Expedia, we have a slightly different context was on the one hand we have micro data so it's super cool we have search level data we see choices. Now, people are confronted with a list of hotels, and they look at them and maybe they choose one so we have to do something a little bit more complicated we're going to do a conditional logic on what thing gets purchased if something gets purchased. So at the upper level, we'll do just a straight logic on the relationship between the decision to book something and the inclusive value from the lower level so it's kind of a four persons fixed. Well, it's a proper nested logic, but estimated in two steps. And so, but again I should mention some some all of these data are in some sense wonderful because they're micro data at the search level. They're also not wonderful for a variety of reasons, one of which I mentioned they're super old but the other data that were given away for this this hacking contest over sampled searches and which purchases took place so you really shouldn't. I'm not sure you should believe everything about the estimates, or at least the size of some of the parameters, but that said, the estimates kind of appear to make sense. In the first column you see for example that people don't like prices. You know, people do like things that are better reviewed and more stars people a little bit like chains, and so forth so that the estimates don't not make sense. The third and fourth columns then are again these rank order loges and here are things, you know, look in many ways like they looked over at Amazon, that the platform likes better products that is likes to rank better products higher. Lower but better sorry, the platform likes prices because it likes revenue, the platform seems to dislike chains. Now the size of this dislike if you look at the third fourth column excuse me the chain coefficient is minus point six. The other coefficients, you know, just think about relative magnitudes that's pretty small compared to the other coefficients in relation to what was going on at Amazon. I'm just trying to put in in our heads and expectation about what the pictures might look like. Okay, so what we can do with the model and can have a little bit of fun, we can calculate rankings, what we can do is we can, you know, take the estimated bias and remove it and then recalculate the what rankings, the would have ensued or whatever risen if the platform had psi equal to zero. So what this picture shows is three things. There's, there's the actual data, which is the solid line. There's the models version of the actual data. This is the distribution of the Amazon products ranks. And then there's the model de bias that is that thing way to the right. If one were to de bias these rankings you'd see that Amazon products will be basically at the very bottom of the heap. Okay, but now, again, I want to issue this disclaimer there there are a lot of issues with these data so this is not so much a claim about Amazon today as it as it is a claim about the the approach. So this is not, you know, this is not don't take this as much in the way of evidence about Amazon. Over at Expedia, all three distributions look quite similar, because after all we found rather tiny bias, but one other thing I'm one of their minor victory lap we might want to claim here is just that model actual and actual actual have very similar distributions, so that it would be distressing if that weren't true. Yeah, so but it isn't not true so that's less distressing. Okay, then then a nice sort of place to look at the results is one of these, one of these welfare frontiers. The Amazon Kindle daily deal data. So the circle is the de biased version of what we actually see. So in some sense, that's telling you how them how Amazon trades off consumers versus producers. And what's a little bit interesting is that it's way over over near consumer land right so the the bias, you know another way to say it. Well, no, let me not do that. Just just think about like the slope of the function at that point, or think about, like what share of maximal consumer surplus is the device point versus what share of maximal revenue. And it's much closer to maximal consumer surplus than it is to maximal revenue. So the point though of course is the difference between the plus sign which is the actual and the device. And so you can see the extent to which actual bias or the actual rankings are in this case in the second bullet for going 3% of the bias CS and 5.3% of the bias producers surplus or revenue. You can also think of it relative to the relative to those 100% endpoints. You can't really see that there are two dots here because there isn't much in the way of bias according to chains and not that we expected there to be so again it's an illustration that shows that maybe if there isn't bias you don't find it. But what is a little bit interesting is that there's again that this thing the circle on the frontier lives pretty close to consumers and farther from from producers. Okay, so I'm going to come in for landing ahead of schedule so lots of time for questions. So where are we, you know, we need ways to test for and evaluate bias and many people in this zoom room are thinking about that and we're thinking all about it together and that's great. This paper presents an equilibrium framework that gives ways to compare testing approaches, and it shows the context in which the outcome based approaches advantageous. These illustrative estimates of the model and that they give us ranked bias estimates platform consumer surplus and producers surplus and the MR the basically the MRS between them, or at least something like that, as well as measures of the of the welfare effects. And I guess we don't need backup slides but I do want to say one thing, which is one of the motivations for doing this, even though we currently, I would say have imperfect data is that the digital Circus Services Act has a provision for vetted access to the kind of data well to some kind of data we'll see in the future, but if one had the right data. It might work very well be possible to implement our kind of our wish list of approaches and get answers that are both sort of transparent but also believable so we're excited about this as an approach more than we're excited about these as results. Because we think there's both a need and an opportunity to study to study this. Okay, I'm going to. I'll stop it there. I'm a few minutes early. That's because I talked fast. It was the coffee. Thanks very much Joe also for keeping so well to time. Now over to Chiara. All right, I can use the sum of those minutes. And then I'll leave the other minutes for questions. Thank you Joel for for presenting this and thank you in candle for for doing this. It's a great paper. I'm literally it's a night to do to use it with with with a project with Andre Prappian and Alex McKay. And it's a great paper because it really tries to bridge the gap between regulation and its implementation, and it doesn't do it. 510 years later, the timeliness is quite impeccable so I think it's particularly important that we have this paper now in the context of the digital markets act where there's a lot of apparently innocuous prohibitions such as no self-preferencing, but then you know their implementation is not as straightforward. So I think the major contribution of these papers is truly methodological and so I want to highlight its value and and clarify, you know, some where it may or may not apply in in the context that we care about. So, the idea I think is pretty intuitive right consumers derive utilities from goods, sellers derive profits from the sale of those goods and these create sort of that parental frontier that you've seen where depending on the way we assign to consumer and producer surplus we obtain different rankings in which those options those goods are presented to consumers right any deviation from this frontier is a form of platform bias so this is where the definition of bias comes in. And this framework is really able to detect and quantify these these bias right. These framework is also extremely appealing, because it helps us make sense of to more reduced form approaches to the tech bias that have already been used in in the literature, the conditioning on on observables approach, and the outcome based approach. And so the first, you know the conditioning on observable approach is pretty simple right you regress rank on observables you have access to and a dummy for the type of products that you want to evaluate bias. And if you have a coefficient on that dummy that's different from zero then you have vice for or against depending on the sign right. The second thing is that it's very likely that you have access to all the product characteristics of I think consumers choice. And so that coefficient can be due right that coefficient on the dummy that you care about can be due to both on observables or bias. Now, I don't think people who have used these approach are using these as a definitive test of bias. I know at least, you know, I have used this with Andre and and Alex. And we're very explicit in that in that very short paper that these should not be taken as a definitive test of bias. So the question that I have for for him can and Joel is whether their model can actually rationalize this condition on observables approaches a first step in the detection of bias. Why do I ask this for two main reasons. The first one is that this is truly the easiest test to implement. You don't need demand, right. You only need rankings and a large chunk of observables, but this is something that can often be straight. You don't need the collaboration of the platforms or other intricate way to collect data right so it's extremely appealing. The second reason is that, of course, this test is not always right, but not all errors are the same. So I would want to see right a better qualification of the errors that these tasks makes as a function obviously of the correlation structure between the unobservables and the dummy right. For example, if it's the case that the test under some correlational structures tends to err on the side of false positives right, meaning that it detects bias when non exist, but when it exists, it detects it right. Well, then it could be a first step in a multi step detection and correction process right and and remember that we can get a sense of that correlational structure between the dummy and the observables why we do have a set of observables right already that we can progressively add to that regression. The second reduced a form approach that their structural approach rationalizes or evaluates is an outcome based test right for which you have you have to have data on demand so this is a total order right. And that's where you know the digital markets act with the with the with the ability of researchers to ask data for these platforms really becomes valuable. And the idea is that conditional on the same rank if one product has lower demand than the other than the platform is faith is biased in favor of that product. Okay, so that's that's the intuition for that test. The second approach actually, both with Monte Carlo simulations and the more structural analysis that in Kendall have done with Amazon and Expedia. It seems to be very closely aligned to the settings to to give very close results to the more structural approach so I, I really sort of enjoy this because, you know you don't you don't need the regulators to make a complicated structural model you can really do do the outcome based approach and be done right. But these brings me to the second point that I want to make which is where these approach can or cannot be a be applied because if we assume that the platform ignores producer surplus right so we don't care about the the profits that sellers make. Then this approach is very broadly applicable. But if we allow for the platform to consider producer surplus, then the assumptions on the supply side are somewhat strong and need for their explanation I think Joe hinted at this, but I want to sort of qualify this because marginal costs are assumed in a simplest form are assumed to be equal to zero and prices set before ranking takes place. That means your price as a function of your product is placed right now, what kind of supply model would rationalize different prices with identical marginal costs. One that I have in mind is a Bertrand model of competition with differentiated products, but here's the catch. When the ranking of prices. In that context, the ranking of prices to be consistent with the rankings of the ranking dependent mean utilities. And so there's wouldn't be a trade off, I think, between focusing relatively more on seller surplus or consumer surplus. There's only a trade off to the extent to which, you know, the high prices are for products that consumers like less. Right. But if the high prices are for products that consumers like more. You don't have that trade off. And, and so, you know, to the extent that there is a trade off I would like to try and have a micro foundation for where that may arise with no marginal costs right and I think this is quite important and see I'm using a little minute, because in this paper the seller surplus maximizing ranking should prioritize products with high prices. But if those high prices come from higher marginal costs, and if Amazon for example, right, at lower marginal costs at a minimum there's no double marginalization or there may be economies of scale, a ranking prioritizing Amazon products because of lower marginal costs right and hence lower prices would be detected by these tests as self preferences when in fact it is not. And so, I think a couple of, you know, these two points one is sort of, can we qualify the errors on the conditional and observables test, and then thinking about where, what kind of supply side can lead to these trade off that you are capturing. But overall, you know, I'm copying you so thank you.