 We're very excited to have Greg Lewis present this really cool work on the welfare effect of consumer reviews. Why don't you take it away, Greg? Thank you. So this is a joint work with Yoga Service, so I bet a bunch of you guys know who's at BU, who's our colleague. And this is work we've been working on for a very long time. I'm excited to present it, but also it reminds me that we really should write up a paper and send it somewhere. So this is good motivation. Thank you very much for giving us that mental prompt. And so what I'm presenting is going to be a mix of older stuff, the newest stuff that we've done on this paper, and all the results that we have kind of seem to line up across many different versions, but they're from different specifications at different points because we haven't collected them to a paper. So nothing should be taken as literally like this is our final answer on a topic. That makes sense. Okay, so let me tell you what the goal of the paper is. The goal of paper is to measure how crowdsourced ratings have affected consumer demand, firm pricing, but most importantly, the distribution of welfare in the hotel industry. And the reason we are interested in that question is because we think that consumer reviews have played an important role in many markets and also illustrate this later. And we have an idea that they affect demand because there have been many papers on this topic and again, I'll refer to those papers later, but Mike Lucas worked, for example, in Yale. I'm sure many of us know. But we don't know whether this is actually doing very much to improve consumer welfare. So just to fix ideas, what are we talking about precisely? We're talking about the star ratings that you'll see, for example, on Expedia or on TripAdvisor. So there's 4.3 out of 5 next to the Shelburne, right? We now know that the Shelburne seems to be a little bit worse than the Courtyard, which is 4.4 out of 5. Does knowing that information really allow consumers to make better decisions and in so doing, improve consumer welfare? Okay, so the motivation here is that reviews are widely used in e-commerce. I started to think of all the places where I think of reviews being used, Amazon Yelp TripAdvisor, Etsy, Expedia, eBay, and it turns out, I checked this out yesterday last night. These are some of the most visited sites on the internet. So this is where people are spending a lot of the time that they spend consuming content. Obviously they spend a lot more time on Netflix, but it's only when I think about shopping, this is something they're doing. And so people are sitting there reading reviews to some extent and we really want to know is this particularly helpful? There's a lot of related literature, which I'm going to briefly mention, but for a long time people spend a lot of time thinking about how the internet was going to be important for price information. So thinking about when this would increase price competition, there was a lot of interest in the study of the so-called law of one price, that the internet would drive prices down to marginal costs. And if the answer to that was just generally no in many ways. And then people started to move on to the question of what about non-price information? I think one of the classic papers there being Jen and Leslie in 2003 had hygiene grade cards being given to restaurants and seeing does that change demand? That's not an internet example, but it was certainly sort of an important paper there. And then papers on book reviews, Mike Luka's work and other people's work on Yelp. And then more recently, Rhymes and Wolf will think about Amazon book reviews, trying to think about the effect of professional reviews of books versus amateur crowdsourced reviews of books and trying to disentangle the differential impact of those two channels. What's going to be distinct about this paper is that we're going to really think hard about wealth there. So many of the ratings papers, the Yelp papers are thinking about revenue. So basically if you get more stars, do you make more money? But here we're going to be very much interested in what this does to consumers. And in particular, we're also interested in a market in which there are going to be meaningful supply-side pricing effects. The hotel market is not a big market. Well, it's not, you know, there are many hotels in the city, but there are not, you know, a hundred or a thousand of them. And so it's going to be the case that people are going to actually adjust their prices in response to ratings. And this potentially is a channel through which welfare effects could become muted. And I'll show you that when we get to the theory. OK, so what's the thesis of the paper? This is the basic story we're going to try to tell. So hotel quality is going to evolve over time. And we're going to have a long panel to measure this. So it's going to be important to think about that. But it's partially unknown to consumers. So consumers have some information about quality, but they don't really know it for sure. Hotel ratings are going to be a signal of quality. And therefore they get effect demand. When people consume this content, read these ratings and reviews, they're going to be able to update. And so this is taking a pretty strong start actually on the role of ratings. The ratings are purely an informative channel here. Nobody's persuaded. Nobody gets high utility as a result of reading a hotel review or rating. They're just informed about which are better and worse ratings. Hotels. Third, equilibrium prices are a function of demand. And therefore there are a function of ratings. And so it's going to be important to think about supply side responses to the change in demand and use by ratings. And therefore, because there is going to be a supply side response, effects on the distribution of wealth are ambiguous. It's not just the case that consumers are all uniformly better off because of this extra information, because it's a game. And as we know in game theory, providing information can lead to everybody being worse off. And so therefore it must be measured using a structural model. There are many things that are not in this paper. And so it's worth spending maybe a bit of time highlighting them. The first is that consumers are making a single purchase decision. And firms are optimizing prices for static profits. And so we're doing everything as though it's static. And in fact, we know that consumers visit hotels repeatedly. And as they revisit them repeatedly, they may become more informed and no longer rely on ratings. We're not going to model that. And firms, of course, maybe think a little bit about the dynamic problem of, if I know that I might have repeat customers, how do I think about charging prices? We won't be doing that either. Maybe more important, I think, is sort of the sense in which we're oversimplifying the whole review process. We're really thinking about these ratings, four out of five stars. But in fact, consumers are going to read text. They're going to read photos. They may be able to improve their welfare by matching on horizontal characteristics. Tripadvisor will tell you what a romantic hotel looks like, which is the best romantic hotels. Maybe that's what you need right now. And that's a meaningful source of value, not the four to half out of five. We're not going to be looking at that either, primarily to simplify our lives. But it suggests that we're probably going to understate the welfare benefits. I mean, the third thing is missing dynamics. So we're not looking at firm side entry, investment, and exit decisions. And one of the channels by which you think reviews might matter and people have written papers about this is by giving customer voice, basically. So by giving them an opportunity to say, I'm unhappy with something, and then having firms respond to it. And this may have meaningful effects on the market. We're not going to really look at that. Entry next in the hotel market, it turns out, is not that important, perhaps, because there's not a lot of it. It's these giant capital investments. And so they're not going to be very responsive to reviews. But investment might be. And we're not going to see that at all. OK, so the modified goal, yeah. There's one question from Jack. I don't know if you want to ask it. Yeah. Can you really study the effect of the ratings without taking into account the way they enter into the recommendation algorithm of the platforms? Yeah, that's a really good question. I mean, I think you can, because what we're going to do is look at the effect on the all-up market of the major review sources. So OK, a couple of responses. So one is that much of demand is mediated through these platforms. So you go on TripAdvisor, you get the rating, you go through the recommendation algorithm rate. But it is also the true that many people will consult TripAdvisor for information and then go book somewhere else completely. And at least from this point of view of the data we're looking at, one of the advantages we have is that the data we're looking at is from STR at Smith Travel Research. And so we're looking at basically a very, very large chunk of the hotel market through all booking sources, not just through the platforms themselves. And so it is true that at least some of the consumers are being mediated through the way that the algorithm reacts to those ratings. But some of them are not. So that's one response. And I think the other thing to say to the extent that these are coexisting, we're just not going to be able to separate them. So if a large fraction of demand is being mediated through the way that TripAdvisor ranks people, we're just going to be looking at the all up effect of how a ratings point translates through the TripAdvisor algorithm all the way through to the demand. Yeah. And yeah, I think you're right. You're also stating an important concern, which is the algorithms themselves, the ranking algorithms are in many ways much more important than the ratings in terms of how consumer attention is directed in general. Okay, so what do we find? We find that hotel reviews have substantial effects on demand and consequently on prices. When we get to prices, you will see that the price effects are not that big and that's because the hotel market is not that concentrated. So there are some price effects that are not as substantial as maybe we thought Exante there would be. Okay, we find the demand effects are big. We find a rating point increases demand by about 27.8% in the final year of our data, which is 2014. I'll show you that there's an upward trend throughout the data, throughout the sample. So one of the nice things about having a panel is we can do this year by year and we can show that the effect of having higher online ratings becomes more and more pronounced year on year. There's substantial heterogeneity. So the effects are bigger for high-end hotels. The effects are bigger for independent hotels. And that sort of makes a lot of sense. You'd expect that where people care a lot about quality, as in when they're looking at five-star hotels, they really care about going to the very best such five-star hotel. And so the effects are substantial. We find that without allowing prices to adjust, consumers are about $2 a room night worse off without ratings, which is about 2% roughly in our data. And when we allow prices to adjust, consumers about 1% or $1 room night worse off without ratings. Although this is distributed over the population. So in our model, we allow for consumers to have differential preferences for quality. And the people who really want high quality are going to be much worse off. They're going to be 6% worse off in the absence of ratings because they're not going to be able to find these best hotels anymore. And yet, so these, I don't know how to think about these numbers. These numbers seem small, but realistic to me. So sort of 2%. I remember having this slightly disparating experience of giving this talk at Princeton once and enlisting people's priors before the talk as to what they thought the number would be. And they were like, yeah, about 2%. And I was like, oh, that's disappointing. It's like, okay, this is not surprising. This is sort of exactly what people think. But hopefully that means it's somewhat realistic. So why haven't we written this paper up and sent it somewhere? Well, we spent a lot of time doing two things that we probably didn't need to do, at least the first one, the second one seems important. So the first things we decided to estimate BLP using maximum likelihood and without additional instruments. And we'll probably switch to GMM pretty soon. But we did that and that took us on a long, long side journey through many, many Jacobians and Hessians. The second thing is we decided to think about capacity constraints seriously on the supply side. So obviously if you're sorry you're good and you've got capacity constraints, your first order conditions are gonna look different because sometimes your capacity constraints will be binding and you wanna anticipate that when you decide how much to price out. And so Chris Conlon and Julie Mortimer have worked on this before, but the models that they have are not very computationally tractable. And so we've tried to work on some different ways of looking at that that are a little bit more straightforward. And so that's something I'll show you a little bit at the end. Okay, so I've got about 30 minutes, 25 minutes maybe. Oh gosh, I'm gonna say 25-ish to go through the data descriptives. I'll tell you a little bit about the causal effect of reviews. Theory instructional model and counterfactuals. I'm gonna skip more than I've said on these slides. I can see I'm gonna run out of time. Okay, so the data is from Smith Travel Research. We have a complete census of about 6,000 hotels in the Western United States. It's about 10% of the US hotel market and about 45% of the hotels report financial performance to STR. So we know all the hotels in the market. In addition, we have this financial performance data which includes revenue, prices, occupancy rates for every hotel year, month. So we're gonna be looking at the Sheraton Palo Alto in January 2014, which is 1.2 billion room nights. And so we have a lot of data here. We know that many observations but we have hundreds of thousands of observations. And we observe a set of attributes for every hotel. And we augment this data set with a panel of consumer reviews from tripadvisorxpediaandhotels.com which were the main three websites during that period for finding reviews. For some subset of the hotels in the years we have daily data on revenue and prices. So at a higher frequency. And that's gonna be important when we talk about the possibility that a hotel might sell out and trying to measure demand in the presence of capacity constraints. And then finally, last data issue. For the structural model, we're gonna use imputation procedures sometimes to complete our data set. So we're gonna match non-reporting hotels with reporting ones to get data on the full set of hotels in the market. That's important because if we only observe half the market obviously competing with 10 other firms looks very different than competing with 20 other firms. We don't wanna use our incomplete data and make completely wrong assumptions about market power. So we use the full census and then do some work to try and fill things out. Okay, so the first time I'm gonna show you is something that I might generally spend a lot of time on but I'm not gonna spend very much time on today. I'm gonna try to show you that there's a causal effect of reviews that reviews are going to matter and then I wanna move very quickly on to the structural model. Great, can I just interrupt you for there was a clarifying question from the audience. I'll just say it, which is one, are you concerned that these platforms might not display all the ratings? They might like remove some of the reviews from their platform for whatever reason. And then two, which is I think a different point, it's been shown that there is like review inflation in certain markets where reviews are going up over time kind of in a secular trend and how do you think about that? So I just- Yeah, so we're in some ways we have this very kind of agnostic and somewhat, I don't know, I wanna say ladies, I don't think ladies are the right word but approach this, which is to say, yes, there's probably a review fraud, there's probably some review inflation, there's probably some review manipulation and all of these are going on and yet consumers will show you are trusting these things more and more. So the market is generally paying a lot of attention to reviews. So I guess one thing you could be concerned about and we'll come to this right at the end is that we're gonna assume the consumers are an average rate. So when they see a 4.4 instead of a 4.1, they have some taste for the 0.3 and additional ratings that they can give out to utils. And on average, they're getting that right. They're not being systematically deceived when they think that they're gonna get more utility from a 4.4 than a 4.1. But we need that to hold on average. So individual review frauds, so small manipulations with edges are not gonna matter. And another way of interpreting the welfare stuff is the coefficients that they place in front of ratings that the sort of preferences for ratings are basically built on how much trust they have in the marketplace. What we're gonna show you is that there seems to be a fair amount of trust in this data even though it may not be perfect if that makes sense. Okay, so what are we gonna do? I'm just gonna show you that reviews matter. And I think the easiest way for me to show that to you is just to show you a bunch of OLS regressions. If I have a little bit of time I might talk about the instrumental variables approach. I won't talk at all about the natural experiment. We also have data from a natural experiment that sort of is designed to prove causality. So the month here is going to be basically a glorified loge at least until I get to the structural model. So people are going to have, there's gonna be demand, which is essentially a market share ratio. So it's the log ratio of my share as hotel J and time T to the outside good in time T where a measure of the outside good is essentially the maximum size of the market in any given month across our entire dataset. We stick in, so that dependent variable is a function of ratings. It's a function of whether there's missing ratings which we then dummy out. We add in hotel fixed effects and we add in market time period fixed effects. So San Francisco in February is a fixed effect. And so really what's going on is if we're trying to identify why ratings matter here, we're asking what is the identifying variation is the within markets ratings fluctuations, taking out a hotel fixed effect. So the same hotel we're really looking at how their ratings vary over time and how demand and co-gurries with it. And they were also taking out a bunch of seasonality at a very fine-grained level because we're looking to take out fixed effects at the hotel at the market month level. You can still of course be worried that the specification, you know, it has fixed effects but that could be endogeneity. It could be the case that ratings are correlated with marketing efforts, the contemporaneous marketing is really driving people to the hotel and then the ratings follows that. So I market aggressively that this is a great hotel. People go there and later they then tell you yes, this was a great hotel. And then we have an endogeneity problem. For the moment, let's just ignore that and I'll come back to you in a second. So what we do is we run this regression and we run it in a bunch of different ways. So we run a straight regression which just looks over the entire data set and asks what is the value of a rating point for essentially a percentage change in demand? It's about 6%. We then cut by management type and we find that chain management hotels don't get quite as much of an effect from ratings as franchises do, which in turn get less of an effect than independent hotels. This makes sense to us because independent hotels don't have the sort of the brand effect behind them. They need more information. We also do this by how fancy the hotel is and STR ranks them from economy through luxury and the coefficients line up in that order. So basically as a hotel gets more upscale it becomes more and more important that it has high ratings for demand and that again makes sense to us. And then finally we have this sort of pattern of coefficients going from 2005 through 2014 and those are again increasing over time to eventually quite a high level, sort of 25%. A full rating point of course is a giant deviation, right? A more realistic deviation is about 0.1 or 0.2 rating points but if you could raise your rating by an entire, your hotel rating by a full rating point that would be a 25% jump in demand. Okay, so that's very substantial. There are some sort of non-minitonisties here we see something to do with financial crisis that was a bit of a dip but in general the pattern is that people paying more and more attention to these ratings over time. Okay, so let me talk briefly. Yeah, sure, very briefly about other ways you could get at this. So we have this idea of using an IV. We have reviews from three different distinct platforms, TripAdvisor, Hotels and Expedia and their importance changes over time. In fact, the main thing that happens is the TripAdvisor becomes much more important in the last half of our sample. It really gets way more traffic. When I looked at the most important websites, TripAdvisor is the top 20 website, expedionhotels.com or not anymore. And so the TripAdvisor's really got a lot more traffic. And so what we can do is we can go ahead and measure engagement through Google Trends as looking at the market platform combination. So TripAdvisor in San Francisco, we go and look at engagement as measured by Google Trends with that phrase over time. And then what we do is we use these two new measures of ratings. So the first is an engagement-weighted platform average. So we're gonna go look across these three platforms. We're gonna use as weights the engagement from these various platforms. So as engagement as measured by Google for these various platforms. And so now your rating at any given time period is gonna be a function of how much people are currently engaging with that platform. So it's important to be doing very well on TripAdvisor at the end of the sample where that's the dominant platform. We're gonna then gonna form an instrument, which is going to be the same, it's gonna be a weighted rating, but now the weights are the deviations from the averages over this time period. So now it's gonna be the case that the instrument says, if you're doing better on TripAdvisor at a time when TripAdvisor is dominant, the instrument is gonna be positive. If you're doing well on TripAdvisor at a time when TripAdvisor is not that important, the instrument's gonna be negative. And the idea here is going to be that the admitted demand shocks, whatever they may be, are gonna be assumed to be independent of the aggregate change in platform tastes. And so as long as I'm willing to believe that whatever this one hotel is doing in terms of its marketing efforts is unrelated to the global changes and preferences of these different platforms, the instrument is gonna be valid and excluded. And so we can then carry through the same scheme but do it as an IV. And what we find is that the IV is considerably less stable. So you'll see that the standard errors are just much, much, much bigger. The first stage is not nearly as good as we'd like, but you get the same general trend in platform engagement over time. The coefficients are slightly attenuated sort of throughout. So we're getting slightly smaller effects from the IV than we get from the LLS. But still big and still on the order of sort of 25% by 2014. Oh yeah, so I moved through that very quickly. Hopefully that was enough to give you sort of the vague sense of what the IV strategy is. Let me talk quickly about the theory and then the structural model. So I'm trying to decide how much the theory did do. Okay, so let's think quickly about this in terms of a hoteling model. So imagine that you're in a world like 2005 where maybe people were not that well informed about hotels. So here I've got two hotels, A and B. Hotels can be high or low quality, but people don't know which switch. And they have priors that it's equally likely that A is better than B, then B is better than A. And the assumption is that one of them is good and one of them is bad. So that world is a completely symmetric hoteling model. The hotels are gonna split the market. They're gonna charge the same prices. The prices are gonna be equal to the transportation cost. And that's sort of a baseline. So okay, so this is one way of thinking about the world without information. Now let's ask the question, what happens when a platform comes to town and tells you which is the better hotel? So once we know the hotel is, say, the better hotel than B, suddenly a hotel is gonna grab, A is gonna grab a large share of the market. And if prices didn't adjust, the threshold type would be given this by this expression here and A would get a lot of the market. All the consumers would weakly benefit, but in fact, only the ones who switch hotels are gonna strictly benefit. So the people who would have bought from A beforehand don't benefit. The information is nothing for them. Likewise, the people who are very close to B and don't switch their decision, but the people between 0.5 and this data bar do benefit. They switch and they now buy from the better hotel. What's gonna happen next? There's gonna be price adjustment. And with price adjustment, hotel A is gonna raise their price because it's commonly known that they are better. Hotel B is gonna lower their price because it's commonly known that they are worse. The threshold type is gonna move back towards hotel A and hotel A is gonna charge more and hotel B less. And so now what happens to consumers? Well, there's a mix actually and it's interesting the distribution of welfare. The people who are near B, who are very near B are actually better off after the information is released that the hotel is bad because hotel B drops their price. They were gonna buy from them anyway because they were convenient and they now get a lower price. People who are even better off are the people who are near the point who switched to the better hotel. They've chosen to switch despite the high prices. We know that they're doing better and they're benefiting from the improved quality. But there are people who are losing in this world and they're the people who are near hotel A who now have to pay more for the same hotel that they would have chosen without their views. And you could think of this as like what happens when it's your neighborhood restaurant and it's beautiful and nobody ever goes there and it has fantastically good quality and then somebody writes up a review in the times and suddenly everybody goes there, right? This is not ideal. And so of course there's gonna be a distribution of welfare benefits. And so that's sort of what we're interested in here. Okay, so how are we gonna get at this when we try to do this using data as opposed to theory? The first step is to estimate demand and that's really gonna be important. We wanna know how it is, we have a hotel model here but we know the preferences are gonna be more complicated. What's the relationship between tastes and prices and ratings? That's the main thing to figure out. Once we know that, we're gonna try and simulate demand when ratings are not known. But in order to get the prices correct, we're gonna have to write down supply side first order conditions and then suppress the information and allow prices to re-equilibrate and see what the market would look like in the absence of this information. So what would it be like if in 2014 we somehow took away the ratings again and people were uninformed like they were in 2005 and then measure the change in consumer surplus across these scenarios. Okay, so this is easy to write down. As I'm sure everybody knows it's hard to do, what are we going to do here? We're gonna have people forming beliefs about quality that they don't know. Their beliefs are gonna be based on rating. Their beliefs are gonna be based on things that are fixed about hotels, like that it has a gym or that it's a four seasons which we know has a good brand. And then characteristics unobserved us which have some notation side. And so essentially what we ended up writing down is a model in which there's a true utility. There are beliefs about that utility and therefore there's expected utility. An expected utility is gonna have a random coefficient on ratings. So there's fixed effects of various forms. There's a larger taste shock. There's some disutility of price but most importantly there's a random coefficient on ratings. And so this is gonna look like a BLP model of demand. Greg, can I ask a quick question from the audience? Yeah, so the same hotel has many types of rooms and they cost a different amount of money. And so how do you think about that aggregation? We, again, we collapse that, right? So we have revenue and we have occupancy and we just go ahead and we said that's the average price. And so essentially what we're thinking about is average rooms that are being sold. And yeah, I mean, hopefully that doesn't bias things too much. It doesn't seem sort of first order again. And then I have, I guess, an additional question. Maybe I missed this. So it seems like you have a random coefficient on ratings but not on price. But your theory model had, well, I guess no. I guess maybe your theory model didn't have a different price sensitivity, but one would think that there are distributional implications here of the price change. I sort of agree with that. Yeah, so I agree. And I sort of think that we didn't do it because we wanted to make our lives simpler, but I could imagine somebody reasonably asking us to do that and we would do it, but maybe not now. Oh yeah, so we assume away the indigeneity of ratings. But we can fix this when we do GMM because we have IVs, so we'll fix that at some point. The main thing that we're worried about is price indigeneity here. And in the hotel market, price indigeneity turns out to be really quite difficult to deal with. The BLP instruments turn out to be pretty weak. The Hausmann instruments are not very good either because it's a geographic market. And so what we do instead is we add a supply side, which we need anyway for our structural contractuals, and we add a particular independence assumption. And let me quickly show you what that is. So in BLP, typically you have two equations that are important. You have an equation for mean utility, which is this delta JM on the left-hand side. And that has an error. This is the unobserved demand shocks. And you have a cost equation, and that has an unobserved cost shock. And it turns out it's sufficient for identification to assume that those two objects are independent of each other. So if we're willing to assume that the cost shock and the demand shock are independent, you're in okay shape. And here we have a lot of fixed effects so we don't feel that bad about it. So, you know, I would generally tend to think that the four seasons is a good hotel. It costs more because it provides better amenities. It's high utility. If I didn't have a fixed effect in there, I would think that those two things were very correlated, but having at least conditioned on the fact that it is the four seasons, maybe I'm willing to believe that shocks to demand and shocks to costs are not correlated. And so that's the approach we've taken here. And I'm not gonna argue why that's true because I don't have that much time. But in any case, you could do this, and we have done this, and it goes badly, at least it did the first time. And the reason it goes badly is because of capacity constraints. So let me talk quickly about capacity constraints because I think this is actually quite important. So if you have three hotels on a market, say, you have very simple loge of demand. They're equally attractive, and they have capacities one, two, and three, and there's a market size of four. Since they're equally attractive, the quote, unquote, first three people to the market are gonna split the hotels evenly. But then hotel one sells out. And the next person who comes in, person four, is gonna have to be split evenly between two and three. Okay? Observed sales are one 1.5 and 1.5. And so if you look at those sales and you ignore capacity constraints, you come to the conclusion that hotels two and three are better than hotel one. But in fact, it's just the hotel one is small, right? They can't take that many people. And so you have to basically correct for this if you're going to estimate the one correctly. Here's also a good point to point out where what we're gonna do is different from what Chris and Julie do in their paper. So what they do when they think about vending machines and capacity constraints is they're not willing to make this continuum assumption that I can hypothetically have a person subdivide themselves into a continuum and evenly split themselves among the hotels. They're gonna actually have literally people walking up to a vending machine and buying a Snickers and then the Snickers sell out. And so that one person, individual person's idiosyncratic shock, they happen to like Snickers, is gonna matter a lot, but it's gonna be computationally kind of a nightmare. And so here we're gonna make this a simplifying assumption that people as they arrive are kind of like behaving like smooth loge of continuum consumers one by one. Okay, so that doesn't work. What can you do instead? You could add data. And so we have data that has daily data. And what we can prove is that you can do an inversion if you know more information. So what are we gonna use specifically? We're gonna take market shares at the monthly level, which we have, but also daily market size measures. So how many people book hotels every day? And there's a lot of variation by day of the week, okay? So with a combination of monthly market shares, but also daily market sizes allows us to tell you how often it's gonna be the case that some hotels are just gonna sell out because there's gonna be a fluctuation in demand that day. And it turns out that there's a unique inversion from the vector of capacity constraints, market sizes, market shares and daily demands back to the mean utilities. And we go ahead and we do that. And then this is a more complicated procedure than the usual BLP inversion. And you'll get to a ton of clever programming to make it work, but you can recover mean utilities. And I can sort of prove that you can do that, but I'm not gonna do that today. And then what I can do is write down an appropriate capacity supply side as well. And the supply side first order condition is going to have to reflect the fact that sometimes raising your price is costless because for some days of the week, say, say Saturdays, if you're a small hotel, you're likely to sell out. So it makes sense that you'd wanna charge a high price. One thing to say about the supply side is we're assuming here that hotels set one price for the whole month in the same way that we think about BLP and car pricing. Of course, hotels are smarter than this. They do revenue management. And so this is an obelisk implication, but it's one that allows us to sort of fit into the stand-in sort of empirical IO playbook. Okay, and then I could show you that if you ignored this, it would make very, very big differences. In particular for hotels that sell out a lot, the extent to which we would get their first order condition wrong could become quite, quite substantial if you ignore capacity constraint. Greg, just a question from the audience, which is how often do these sell-outs happen? And then a follow-up question from me, which is like in the data, it's not always, it's a kind of a judgment call when the sell-out occurs. So how did you choose that judgment call? Yeah, both good questions. So hotel sell-outs are not, so first of all, why do you have to answer your question? So we'll often say that the sell-out to 90% is a sell-out rate, it could be 95, right? And then there's a question of like, okay, and so how often does it happen? And we're finding, I actually can't remember the number of my top of my head, but I think it's sort of like on the order of like 5%, maybe it doesn't happen super often that data. But importantly, we never actually measure sell-outs. What we do instead is do this all through the structure. So because we're just using the daily data, total hotel bookings, not individual sell-out rates, we never have to make this judgment call. We just have to know how big the market fluctuations are to rationalize our monthly market shares. But we've also done some separate work in trying to figure out sell-outs, and then it's different. Okay, we estimate the model by simulated maximum likelihood. We run monocalls in a different paper that showed this is statistically efficient. It is, it's great, it's just really a pain. And then we plot the distributions random coefficients. And so what I'm showing you here is that over time, it's more and more the case that we're seeing increasing tastes for ratings, people care more about them, and also sort of a dispersion is increasing. So how much people care about them is changing over time, or that might be a little bit of functional form. Okay, finally, let me talk for the next like two minutes about kind of actuals. So what are we gonna do? We estimate all of this, we estimate demand, it takes a long time. We get a pricey list to stay estimate of about minus two. And then we compute kind of actual national prices. Here, as I mentioned before, we have to deal with sample selection. So we do some imputation. And then what do we do in the actual kind of factual? We say, suppose that instead of knowing the true rating, instead people had, they thought that all the hotels in the given market had the average rating of hotels in that market. So collapsing the 4.8 and 4.2 of the two hotels in the market said a 4.5. And then saying, what would people do when they just thought it was 4.5? And we're gonna do a couple of scenarios. We're gonna do one where hotels keep the prices the same and another where they set them in complete information national equilibrium. As I said before, the change in consumer surplus is about $2 per room night if they stay the same. If they set them in random, in complete information national equilibrium, it's gonna be the case that the really low rated hotels are gonna charge more. The really high rated hotels are gonna charge less. The effects we're finding are in fact almost too small. And so that means maybe since the structural model doesn't fit perfectly, they're not big though. They're on the order of $6 up at the top and $6 down at the bottom, which maybe says something about how much market power individual firms have. Nonetheless, this scenario has very big effects on revenue. So hotels at the bottom would make, we estimate 50% more revenue if the fact that they were bad hotels was suppressed. Hotels at the top and there are way more hotels at the top would be losing sort of 11% in revenue. So it would be a substantial hitch to their revenue. So these ratings are important, but how much they change where people stay and therefore the distribution consumer surplus is much, much smaller than the individual impacts on firms. And so just to conclude there, we find that in a kind of actual world without ratings, the average above average hotels, four-stop plus, they're gonna lose a market share. They're gonna decrease prices. Well above average, below average hotels are gonna gain market share and increase prices. And the effect of consumer surplus is about one to 2%. It's much bigger for firms. It doesn't appear to be to do massive amounts to markups. That is to say the ratings themselves are not quite, the ratings changes are not quite big enough dramatically change markups for the firms. And then finally, heterogeneity is important. So consumer surplus effects are big for the kinds of people who care very much about quality. So those go from sort of one to 2% up to 5, 6%, which makes sense. Okay, so that's it. I wrapped up the last button about three minutes, right? Thanks for that paper. And I think now Kiara is discussing. Yep, I hope you can hear me well. Thumbs up, down, yeah, okay, perfect. So thanks for giving me the opportunity to discuss this paper. I really like this paper for a few reasons. The first one is that this is really the first paper that takes seriously the estimation of welfare effects from online reviews. And yes, there's been some follow-up papers in K. I think Rhymers is in the call today. She and Joel will follow up a paper on book reviews. But I would say that these are these hotel paper proceeded by a few years. The graph was very upfront about the fact that the model is static. They are thinking about maximizing static profits, setting prices statically and having consumers making one-off decisions. And the fact that the underlying quality is sort of cannot be altered by hotels. The other piece that I really like, given our common work, is the fact that they take capacity constraints very seriously. And because these has implications on sort of mean utilities that get estimated for hotels that sell out often and hotels that don't sell out but get remaining residual demand because of the sell out of the other hotels. And in terms of results, I think they make a lot of sense because ratings lead to better matching between travelers and hotels. Travelers are better off on average. In particular, the benefits accrue to travelers who care about quality. And just like a very simple back of the envelope calculation and I'm not sure it's actually completely correct, hotels sort of capture about half of these extra surplus and travelers capture the other half of these extra surplus. Okay, and the other thing is that ratings are more influential for independent hotels for which consumers won't have strong priors. So also these makes a lot of sense. I have a couple of questions. The first one is really about what are we capturing and whether we can separate the role of reviews from the role of the internet and everything else that these online travel agencies are doing. So if I think about online travel agencies, I'm pretty sure that demand for hotels would have changed given OTAs even in the absence of consumer ratings. And that's just because consumers can find hotels online that they didn't have otherwise planned. Okay, there was a reduction in search cost. And so is there any way to separate really the role of online ratings? And these has to do with the type of counterfactuals that you want to run. The current counterfactual has hotels having the same average rating in the city and month, I believe, but maybe we should make the average rating be dependent on some observables that consumers have even in the absence of that five star review. So for example, there's an expectation for ratings for chain hotels that is different for the expectation of ratings for independent hotels or for different locations within a city. The second thing that I wanted to focus on was the fact that benefits are overwhelmingly concentrated among consumers who really care about quality. And if I've done my simple math correctly, if the top quantile, and I think it's quartile, enjoys the most benefits, that means that the other 75% of consumers have negative welfare effects. And so it's super important to really look at the distribution of the losses and gains that these ratings have affected. And here's why I actually think, and it's something that Jacques mentioned at the very beginning, is it reasonable to think that 75% of consumers get negative benefits from ratings? Probably not. And here's why the ratings are bunched up all the way between 4.9 and five stars. And so I think the ratings, as stated, don't signal the heterogeneity that actually the heterogeneity of information that consumers can receive from the internet. And so that's why I think in a sense, you're underestimating the benefits of ratings because of these rating bias. Okay, and the last thing is, if you can talk a little bit more about the imputation or how do you think the imputation is gonna affect especially your counterfactuals on independent hotels? Because I recall reading that independent hotels are much less likely to share information with Smith's travel research. And so are you worried that these imputations may sort of bias upward the benefits to independent hotels because maybe only independent hotels who are successful are disposing information to SDR? And with this, I'm done and overall, great. Shall I respond, Andrew? Yeah, okay, great. So thanks very much, Kira. Those are fantastic comments. I mean, I think so your first thing was about what's really being measured here. And I think my reaction to that is maybe that you're saying there are these important secular trends that we're kind of not accounting for, like maybe independent hotels are becoming important. Perhaps we should consider trying to think about how we could take account of those in the demand system and see what's left over as residual variation that correlates with ratings. So basically, rather than allowing ratings to just be this thing that moves over time. I mean, in some sense, our defense would be, look, within independent hotels we have variation ratings in part were identified with that, but would it be more convincing if we were able to sort of say, okay, allowing for a trend in the direction of independent hotels say, what's happening there? And so I think that's one thing we could do for sure. Your second question was about the distribution of gains. And I think that goes back to the same thing I mentioned at the beginning, which is there's so much more information in reviews than is being captured by the rating, which I think we're saying the same thing. It's sort of like, it's gonna be an underestimate, it's gonna be an underestimate because we're being too coarse with how do we think about the world? And so, yeah, I think one, it's a great suggestion for us to say, this is what we find, 75% lose, 25% win. This is, and then to say, but maybe we're getting it wrong. And I think both those things are important. I think the decomposition of who's winning and losing is very important. And then also to say, okay, but there are these caveats. So I would add a hundred percent agree with that. And then the final things you mentioned, the imputation. So, I mean, yeah, we're doing pure selection of observables, right? So to the extent that it's their specific independent hotels that show up in the day to do better, they are gonna get overweighted. I'm not sure what I can do about that unless I can find some more intelligent imputation procedure or something that uses an IV to check. But it's worth it to be thinking about or trying to do some contest on. I don't know what that right solution is there, but that's something for us to think about.