 Thanks so much for having me. I've always been a big fan of this. Yeah, so this is drawing work with Apostolos and Diego. So let me just jump into the context because it's kind of important to understand what's happening here. So this context, it's a large online labor market for work that can be done remotely. So you should think of things like computer programming, graphic design, projects here very dramatically. And some could be just a few hours long and others could be year-long engagements. And it's very sort of unclear to workers here sort of how big these jobs might be and certainly unclear even to the platform when they're posting. How matching happens on this platform, it's somewhat out of the control of the platform. So it's not like say an Uber where they're kind of dictating pinpoint matches. But an important way that matching happens here, which just kind of has parallels to conventional markets, an employer will post the job description saying what they want done. They'll get an algorithmically generated list of workers that might be suitable. And nothing too sophisticated, just kind of matching on keywords and kind of the skills. Employers can then inquire whether those workers are interested in the job. And then the worker can choose to apply or not. And then the employer can decide who to hire if anyone at all. A key problem in this market is that these are individual workers. They only can work some number of hours per week just due to the kind of inherent nature of work. And so they have this problem where they can kind of inherently, they stock out and, you know, turn down recruiting employers and say, oh, sorry, I'm not interested. Now you might think, well, the platform can try to take and make predictions about this probability of turning down an invitation and they do and have for a long time. But it doesn't seem to work that well. And the kind of a key claim of this paper is it doesn't work that well, primarily for economic reasons, not really technical reasons. You know, this is obviously the context here is a particular online market. But this is like not having sellers who are inherently supply constrained is not that unusual. You know, real estate, if we if you ever participated in the real estate market, you know, you typically have one house to sell and you're only looking to buy one house, labor markets, obviously, but even like a lot of, say, quote unquote, sharing economy type companies have this markets have this kind of feature where, you know, you have a kind of an atomistic seller that's got only one unit of supply. And the buyer might not have much insight into whether or not something is even like on the market. So, you know, why does this matter? Well, if you recommend an unavailable seller, a match can't happen. So, you know, a house that's already been sold, a candidate who already has a job and is not looking an Airbnb room that's already booked. And if you kind of think about it, in terms of like recommender systems, a lot of the things that you might think of as like the first heuristic you would have would make this problem worse, right? So, if I'm Amazon and I see that, you know, people like Harry Potter books, and I'm kind of dating myself, I don't know if the kids are still into that. But like if I see that like a lot of people like a particular book, there's really no reason I can't recommend that to almost everyone if it really is very, very popular. If I try to pull that same stunt in a labor market, like, Oh, well, Dave builds good websites, I'm going to throw the fire hose of website building traffic at Dave, he's going to quickly kind of stock out and be unavailable, or, you know, maybe just raises prices to some sort of ridiculous, ridiculous price. So, you know, I to be honest, and this is not a problem that I have only thought about recently. This is like one of the first papers I wrote as an assistant professor. And just this paper was published and said, Hey, you know, this problem is pretty bad, it causes a lot of loss of efficiency in markets. But it also talked about a potential solution. And the solution, or to say the cause just is that the seller's capacity is their private information and they're not too keen to reveal it. I show that this matters for matching. And then I talked about a partial solution that really wasn't mine, it was the platform solution. And the solution was, well, why don't we just ask sellers about availability. So prior to this, there was no question, like it just was sellers weren't sort of asked to update anything. Well, and so the idea was really simple, you can just say to freelance workers in this marketplace, hey, what's your availability, and they could say as needed or open to offers or, or, you know, I'm full-time, I'm 100% available. And the idea was that this would just get reflected back to potential buyers. The buyers would see this and condition on this information and help direct their invitations, their recruiting invitations to sellers that had more capacity. But nine years later, I would say that this was a temporary bandaid at best. This does not actually work that well. And let me just show you what happened. So if you think that capacity is somewhat dynamic, you would expect sellers to be changing it somewhat regularly. Back in 2019, about 5% of sellers were changing their availability status each month. Over time, that's plummeted. It's basically moved to a world of set and forget. And worse than that, almost 90% of sellers now state they are available full-time. So no one changes it. And most people leave it saying that they have full capacity to take on more work. Despite this, only about 30% of buyer inquiries are even responded to by sellers. Now, I mean, maybe those seller inquiries are bad. And that kind of makes things kind of econometrically difficult to know why precisely they're turning them down. But it kind of suggests that there are probably a lot of people that don't have capacity, but are saying that they do. And really, the core issue, and this is why I think this is more of an economic issue than anything else, we all know that job offers are valuable, even if you're not going to take them. You can use them to negotiate. You can take and pass on to others and try to reciprocate. I mean, not literally the job offer, but you can just kind of point out like, oh, here would be another person you might want to consider. So if I'm a seller and I can get more offers at zero cost just by saying I'm more available, then I will. So the problem here is this information, this private information about seller capacity, it's missing for economic reasons, not technical reasons. If it was just a matter of asking and formatting it nicely, this would be a much easier problem to solve. But that's not why. So the question is, how can the platform get sellers to reveal their private information? Well, the idea here is, well, let's make this cheap talk expensive by introducing costly advertising to the platform. And it's essentially makes sellers pay to say they are available. Now, in the paper, we've got a formal model of why this might be a good idea, but I think the economic intuition is pretty clear. If the problem is that saying you're available causes you to get more offers, well, if you raise a price for saying that, then you're going to select for people who are actually more available and might have more capacity. At least that's the hope. So people will quibble about whether or not this is advertising. I think it is, and I think it fits in an IO tradition, viewing advertising as largely about signalling. And so one of the quote unquote good views of advertising, just beyond the obvious one of signalling like, hey, I'm in the market and I'm willing to trade, is that it might send information about latent quality. Now, in this case, quality is not quite what we're talking about, but it's kind of closer related. That being said, economists have had fairly mixed views on advertising for other reasons that it might help manipulate preferences or increase costs. And there's also this argument that, okay, well, even if maybe there is this good informational value of advertising, maybe digitization has undercut this rationale. So there's an article in the Yale Law Journal a few years back saying, well, look, we've got Yelp and all these product review sites and everything else that, is there really the informative view of advertising? Does this carry much water anymore that basically it's all about manipulating preferences and it's not that useful? Well, so what this paper does, so I kind of described the broad context, it's about a field experiment that assesses this informative digital advertising view. And the practical goal or the applied goal is to solve a somewhat tricky market failure. So the implementation is pretty straightforward. Sellers on the platform were given the opportunity to pay P dollars per week. It was not actually dollars, it's on platform currency. But when they do, they get this badge added to their seller profile. Okay, so if I'm a buyer now, I can see a little badge that says available now. If I mouse over it explains that this is a paid promotion that the seller was paying to have this appear. And the platform, why did they even test this? Well, the hope, the positive view was that maybe more available sellers are the ones who are willing to advertise. And that buyers would seek out those sellers because they know these sellers will be more responsive. And so in equilibrium, we essentially induce a separating equilibrium in the market. But the fear, and you know, this is not like a fear that only economists would raise. It's a fear that you would see in the forums and lots of like product managers and designers kind of brought this right to the fore, which is, well, what if desperate or worse sellers are the ones willing to advertise? And they're higher, higher available, they're more available for essentially a bad reason. If that's true, this isn't really going to work in equilibrium because buyers are going to learn to ignore or actively avoid advertising sellers. And in equilibrium, there'll be no pooling and no one advertisers. Okay. And I think that it's, I want to take these fears pretty seriously because you kind of hear this about sort of platform advertising more generally that our sellers adversely selected. And even going back to Nelson, who's kind of, I think a really important paper in the econ way of thinking about the signaling, even if it wasn't that formal, he kind of makes this point, well, look, if you think of a competitive market, the sellers that are advertising, they have higher costs. You think that in equilibrium, they would be the worst deal. So you'd be like, you'd want to avoid these. And this logic is out there even in kind of big successful reef tailors, the Costco CEO, they famously don't advertise. And he talks about this that he says he considers the evil that basically just raises costs and should be avoided. So anyway, to summarize, there's kind of like this lemons and peaches argument here, the lemons argument is, if you're so good, why do you need to advertise? If you want something done, ask a busy person to do it. The peaches argument is buyers don't like pursuing unavailable sellers. And there's lots of reasons someone might have low capacity at a moment in time. And there's lots of hidden information in markets that advertising sellers might have just for whatever reason, a greater upside to doing more work and it's not a bad thing. So here's this kind of like interesting, I think, tension that kind of frames why it's worthwhile to do an experiment. And I'd say, I primarily identify as a labor economist. I think most labor economists, the lemons argument would be to the fore. There's a long literature showing that employers are predisposed to not like sellers, in this case, workers who have been out of the market for a long time. If you have a lot of capacity, they think that there's a reason for this. And so there's this kind of like strong hysteresis and strong path dependency in unemployment that labor economists can worry about. So the platform proceeded pretty cautiously. It let sellers in a select technical category of work by this paid advertising. And then they randomized buyers to being able to see that advertising or not. And then they can observe what happens. And the idea was like, well, we'll adjust prices to try to optimize how much information, if this thing doesn't blow up in our faces. So take up of advertising was really rapid. So about 40% of eligible sellers opted in within two weeks, which was good. And then the random, so the sellers, there was no randomization on the seller side. It was just you were available and you could, you would get a notice about it. But there was randomization on the buyer side. And how it worked was that if I was in the control group, I would not see any ads. So I would get the same status quo experience that I had in the past. If I was in the treatment, some of that real estate was now consumed with this little badge that said available now. And if I must over, I could see that it was a paid promotion. Critically, nothing else changed. So there was no change in the assortment. There was no change in ranking. There was no change in sort of the size of the tile. And, you know, this is actually important, you know, Andre and I have a have another paper where, you know, we were, we were looking at basically doing some informational interventions around sellers being presented and in a mark or rather as it was reversed. But, you know, even just kind of subtle changes in sort of how much real estate you give things can kind of change the search process. Here, there was, there was nothing, it was exactly fixed. So the experiment itself, you had about about 40,000 buyers in the treatment and control conditions. Well balanced. The platforms use this experimentation protocol lots of times. So no, no big issues. It was all happening in the same market. And so the so for condition here is like inherently violated. Now I'll talk more about that that later. But everyone is kind of participating in the same market, getting exposed to the same sellers. The assortment of sellers they get was identical across the two. And we have a lot of kind of things we can look at internally to see like, okay, they're well balanced, but they are interacting with the same sellers potentially. This is allocation happened when they post a job. So, you know, critically, this is a presentation that happens when you post a job and you get this first collection of recommended sellers. This is where the action is. So I have a number of questions that I want to look at. One just is, does this, do buyers actually seem interested in these kind of sellers? What does it do to the inquiries they send? So does it just shift out who gets invited? A shift around who gets invited? So I might send the same number of invitations, but now I'm drawn to the treatment or if the lemon's argument is true, maybe I'm, sorry, not to the treatment, to sellers who are advertising versus not advertising. And then what does it do to matching overall? Because, you know, the ultimate goal here is to try to improve matching efficiency. So one thing that's nice about this online context is we can look at the level of the impression. So a particular buyer seeing a particular seller. And with that, we have about three and a half million impressions. This is from our allocated buyers that are searching. You can see about 50% of the impressions the seller was advertising. And only about half of those with a buyer know this, that depends on their treatment assignment, but there was lots of sellers advertising that they were, that they were seeing. Oold overall impressions, buyers send inquiries to a little less than 10% of the sellers that they're exposed to. It's about, it's about 8%. So a lot of, you know, most of the time they're kind of just blowing past people and not inviting them to apply. Let's compare inquiries by seller advertising status, position, and buyer treatment. So this is just a seller's position in the buyer's search page. So one is the highest ranked, go all the way down to 10, and it's paginated that way. And this is just the probability of receiving a buyer inquiry. So a buyer inquiry here is just that they smashed that invite button and invited them to apply. And then let's just first we'll look at advertisers, sorry, non-advertisers. So these are sellers who are not advertising. Treatment buyers can see ads, control buyers cannot. And we see essentially no difference and it holds true across the thing, which is what we would expect. These are all sellers who are identical. They're not advertising. They should look kind of identical to treatment and control. There's some ways in which that might not hold, but this is a useful first kind of step. Now let's look at when advertising sellers get the impression. We can see that the control is consistently below the treatment. So when exposed to advertising sellers, these treated buyers seem to seek them out. So these are all advertisers. It's only when you're in the treatment you can see it. These are the people that you're more interested in. Another way to visualize this and maybe a little more straightforward is actually to group by buyer status. And so when we see this, when the buyer is in the control, you can see that they show a slight preference for non-advertisers. So they can't see anything, but they take team to prefer those sellers who are not advertising. They just look better and something about them on average is making the buyer more interested in the non-advertisers. And essentially what happens when we go to, when the buyers can see it, we more or less close this gap in a little bit more. So what happens is now when buyers can see the status, those sellers who are advertising get a benefit that kind of brings them up to parity or maybe even a little bit more. Okay, so it does seem, based on impressions, that buyers seek out advertising sellers. Why? I'll get into that in a bit. To actually get to the second question here, what happens to inquiries in total, let's move to the cumulative buyer experience. So remember, buyers here are the unit of randomization, not impressions. So I'm going to estimate a post-on regression where I'll look at the total number of buyer inquiries sent conditioned on whether or not they were in the treatment group, whether or not ads were visible for them. Okay, and so what we see is they sent about 3% more inquiries in total. So this is not just to, I'm not conditioning on the seller status, this is just looking at the buyer level, those who are in the treatment send more inquiries. If we look at the count of inquiries specifically to advertisers, we see much, much larger effects, as we would expect, about 6% more. But interestingly, we don't see a measurable decline in inquiries to non-advertisers. And this kind of surprised me. I think I kind of had maybe a lump of inquiries view that this was just going to sort of shift it around from one to the other, but it actually turned out to be there was some kind of action on this intensive margin of how many inquiries I send. Now, so it increases the number of inquiries, no kind of direct evidence of crowd out. Does it actually work? Does it lead to more inquiries overall? Well, we actually see about 3% more proposals that are received. So this is the actual buyers taking and then sending, they send an inquiry and then they get a response from a seller. So we get more of that, which is what you need, like that's kind of the grist for matching to happen here. And then finally, contracts formed, we see about similar increases of about 3% more contracts formed. So, you know, subject to the caveat that this is all happening within one experiment, we've got some evidence of more matches being formed from advertising. Well, okay. Why does this happen? What's the reason? Why did buyers seek out advertising sellers? Well, you know, we kind of have a theory. It was that they looked more salient or they made one theory is they look more salient. Another is that buyers made some inference that these guys might be better. Well, one thing we can do that's kind of unique to this context is we can actually look in and see how did the sellers differ by their advertising status, right? And we can look at what they had, what they were doing before they were allowed to advertise and what they did after. So first to start, we can see that there's really no evidence that they differed in their stated availability at all. So this is kind of like the core problem. Advertisers and non-advertisers both said they had full capacity. But if we look at some other measures on the platform, which people would associate with being like a good seller here, advertisers have more complete resumes. They have higher feedbacks from past contracts and they're asking for slightly lower wages. And these are all pretty sizable differences. Advertisers are also on average getting more inquiries and responding more positively to buyer inquiries. So they really do seem to be more available now. If you look at their response rate, it's almost 10% larger. If we model selection into advertising just statistically and took kind of like a number of kind of predictors, this is all sort of pre-experiment attributes. We can see that basically sellers who are not getting as like the fewer inquiries you're getting, the more likely you are to advertise. The more number of bids you're actually placing follow buyer inquiries, the more likely you are to advertise. So these are people who, conditional upon getting an inquiry, are they actually likely to follow through and send an application? And then these are just kind of these virtuous signals of like how many contracts have you formed, number of bids placed, all these kind of indicators that are factors they could control. They seem to have good reputations. Okay. So I think that this kind of naturally lends itself to a pretty simple interpretation. Advertisers want more business and they're not getting it organically. And so advertising lets them attract more buyer interest, which they can kind of profitably use to turn into more business. So I think it seems like to treat a buyer seek out advertising sellers because they seem to be better and are more available. Now, I mean better presumably gets factored into prices, but this available thing definitely seems like something that buyers would have had a hard time conditioning on. And this is the information that's kind of getting revealed by this. Maybe advertiser sellers change their behavior or prices. So one thing we can do, it's not exactly rock solid, but because we have sellers observed over time, we can look and see what, you know, we can create a seller panel and interact with the experimental period with how many days they were actually advertising. And, you know, we can kind of see there's really no change in their bidding behavior. So even though they were advertising, this didn't get turned into a price effect, which has kind of been a concern in the literature with maybe people advertised to get like a modicum of market power and it just gets turned into a price effect, but at least in the period of the experiment, that doesn't seem to be the case. Okay, but what about in equilibrium? Well, the platform then introduced this broadly, but only in one category to start. So all buyers in the technical category were moved to the treatment. The platform annoyingly raised advertising prices and changed availability two weeks before why to roll out. So the price increase reduced the number of advertising sellers somewhat, but those left were likely even more available. So the idea is maybe we can compare treated category to the largest comparably sized non-technical category with minimal spillover and crossover of applications. And, you know, I mean, frankly, I don't think this is super compelling yet, but we're kind of still working at it. But there seems to be some evidence that we get like some of the treatment effects kind of hold even in equilibrium. And we're working on analysis now where we've got kind of a cleaner change in prices that will kind of maybe let us disentangle this a little bit more. But if you kind of look, this is going through the number of openings that doesn't seem to be a change. This is the number of job posts. We do see slightly more inquiries, which is what we would expect. We see more proposals from inquiries, which is what we would expect. And we see some some evidence of more contracts being formed, which is what we would expect if the experimental results were more generalizing. Let me just like step back for a minute. And this is probably, you know, of interest more broadly to people kind of studying digital marketplaces or platform economics. There's a lot of evidence that digital marketplace advertising is growing. A lot of companies are kind of adding this to their repertoire of ways of making money. And, you know, I think it's in some sense understandable because there's really two marketplace problems that it can help solve. Nearly all marketplaces face some kind of visibility allocation problem, which you think of as what sellers should buyers see. And, you know, there's a mirror image of what buyers should sellers see. There's some platforms that obviously take this over and, you know, 100%, you know, even determine matching. So if you take like the ride sharing context, this is really not not an issue. But almost every others has to make some decision about who they should, who said they should show. I would say this has mostly been addressed algorithmically, taking an information retrieval approach to ranking personalization recommendations, etc. But I think a flip side of that is you kind of recreate all the Google SEO type problems internal to your marketplace. Because, you know, given the stakes to sellers to being shown, you invite lots of gaming and manipulation. Where it becomes even if you have maybe what starts as a very good algorithmic approach to the problem, you can lead to lots of adversarial behavior. The second problem is that all marketplaces have to raise revenue. And if you think about kind of the in broad strokes, what are your options, if you have fixed fees, you're going to screen out lots of low value transactions that the platform could profitably intermediate. And if you have ad valorem fees, you can screen out low margin transactions, the platform would still like to intermediate. And so, you know, you and if you set whenever you set these prices, you always have this problem that, you know, they could be too, too high or too low relative to what the market could bear. And so advertising, potentially is a nice solution to both problems or at least a partial solution to both problems. It's a way of solving this visibility allocation problem like who is shown is determined through a market system, right, as opposed to just using something entirely algorithmic. And it has this ability to raise revenue. And it seems to be particularly well targeted in the sense that if you have sellers who have a high willingness to pay, you know, they're the ones who are actually paying what amounts to higher fees. So, you know, I think I'm, I think there's a kind of a number of interesting follow on questions that I'm very curious about beyond just looking at, okay, you know, a year in how well is this continued continuing your work. One is just what's the optimal price for this kind of advertising. And you can if you think about the extreme cases, you can see that neither worked that well. So if everyone were to advertise, I see, say you kept the price very, very low, you know, it's not informative. And this is more or less the equilibrium that prevailed before it was free to say you were available. So almost everyone did and it just didn't have any effect. But if you set the price too high, if no one advertises, that's not informative either. So, you know, like maybe it's a half, like that's that's the quantity you would want. That kind of has some intuitive properties that that seems like it would maybe reveal the most information, but I have no really strong theoretical reason for thinking that. And so if say you have some goal of maximum maximizing the number of matches, is there some principled or market based way to do this? And I can tell you like what the platform does now is, you know, periodically have a meeting and kind of decide prices. And, you know, as an economist that that feels that feels wrong, like there should be some better way of figuring out what this price should be. Another question that I think is is interesting is, is the optimal message space here finer than binary? So here we just said available now or not. And if you kind of talk to qualitative researchers on the platform and even sellers, you know, they had much more complex preferences that they would like to communicate, like I'll be available in two weeks, or I'm somewhat available right now. But you know, I have a hard stop in, you know, a month, or I'm available for this kind of job or not that kind of job. Well, you know, if you allow all the permutations of availability someone might have, you clearly are cutting against the simplicity from a buyer perspective of sorting, right? If I have to go read through a long free text where you kind of narratively describe your availability, how am I going to actually be able to kind of direct my my search? So there's clearly like somewhere between free text and binary is probably some kind of optimal message space. And I think that's kind of an interesting question. A third question that I think is interesting. So as I said, the platform during the experiment kept the assortment exactly the same. They didn't try to incorporate the signal into ranking. That being said, it would be a very powerful, useful predictor in any ranking algorithm. So if you were able to use the seller status to rank people differently, you could kind of dramatically improve the quality of your recommendations because you'd be recommending people who are, you know, 10% more likely to respond. The challenge is I'm not sure how you actually could do this in a principled way. Like how can you sell movement along a gradient? Because, you know, ultimately if you kind of putting a price out there and saying, hey, if you pay this, this is what happens, you know, it may not work that well for you as a seller, but at least you understand what's what you're buying. Selling like I'm going to give you magic ranking juice if you give us money is kind of a weird position for the platform to be in. So I think there's kind of an interesting question of if you want to take and improve your ranking algorithms by using a paid promotion fed into that system, how would you do it? Or should there just be a firewall? Like you should just kind of keep these two ideas completely separate. The last is in some sense, we got a little lucky here in that advertising sellers were the kind of sellers that buyers seem to be interested in and this availability dimension really mattered. But you could easily imagine that in other platforms, that's not the case. And, you know, Sarah Moshery has this great, great paper that has a lot of similarities to ours that really has a different result that that advertising sellers were adversely selected. But the platform was kind of weighing off. They made more money from the advertising than they did from the what they lost by somewhat degraded matching efficiency. I think from a platform perspective, I mean, you could you could just kind of hope that you get lucky here. But I think an interesting design question is how do you make sure that your advertising sellers are virtuously selected? And, you know, it kind of put it in a pithy way. Like, you know, can you make the good rich as opposed to just hoping that the rich are good? And I mentioned this, but one of the things that the platform did here, it wasn't directly money that they were paying for advertising, they were using an on platform currency, which in the paper, we call coins. And what's nice about that is you can potentially do airdrops of coins to sellers who do good things. So if you create a good experience or get a feedback that's positive or anything else that's positive, I can give you more of these coins and effectively subsidize advertising for you. Well, you might say, okay, well, why not just do that with money? In any real kind of platform context, as soon as you start kind of giving out money for doing things, you're going to open the door to an amount of fraud and scams that, you know, there would be no end to, right? So kind of, it's nice to have your own platform currency and being able to give it away where the only cost is kind of the suppressed purchases of that. Those coins in the future is a way, I think, to kind of shape the platform in the direction that you would want, that sellers kind of stay virtuously selected. So I actually finished quite a bit early, but that's okay because I think the discussion part is probably more useful, more interesting anyway, but... Thanks, thanks, John. In our, why don't you go ahead with the discussion? Okay, do I'm going to stop sharing? Yes, please go ahead, if to write your... Okay, hopefully everyone can see my slides and some reasons. Okay, so thanks so much for having me. I'm also a fan of this particular seminar series and of course, also a fan of John. John's work has, I think, advanced or understanding of online economy when it comes to the labor markets quite a lot and this paper seems to be also going in that particular direction. In this paper, I mean, John has gone through a lot of details to sort of explain everything under the good job, but I think just to put to overall importance, there's this big question about how we can potentially increase the matching efficiency, efficiency in matching in labor market. And of course, in labor markets, an important source of frictions is really coming from a lack of information, information about the characteristics, the quality of the participants, sometimes about the other party. In this particular example, the friction might be the availability information or perhaps, but I think more importantly is the capacity of the workers to be able to take on more work because that seems to be where I think the results are mostly coming from. And that's what they are focusing on, what they call advertising. There's a minor question about whether this actually this type of information, increasing information about the availability and the capacity of the workers, whether that counts as advertising, I'm sure a lot of people in the marketing side would ask this question. And I think it is a form of advertising because it's essentially something that you select into as the seller. You choose this characteristics and you pay for it to be able to make that information known in the marketplace. But nevertheless, it's a different kind of advertising than some of the types of advertising that we are used to seeing. And John and Paul, there is run an experiment essentially to see if this particular type of advertising might do anything in the marketplace because its outcomes are not necessarily no. So what are they finding or how could this type of advertising of availability might work? When I think about advertising availability, just like most other people who might be thinking about the potential downsides and upsides, I think that it could do multiple things. First, there is this potential negative signaling or adverse selection where who is going to again think about putting a sign up there that says I'm ready, I'm available, I have nothing else to do. This might indicate lack of work for at least some workers. And it might come from again, the fact that they are undesirable workers, they might be the those who self-select into advertising might be the ones who we should not perhaps be hiring. Or it might indicate that maybe there are some workers who are quite capable, but there's enough work for them. There's maybe a decent supply of these workers, but there is fairly low demand or there's a supply and demand imbalance in this particular marketplace. That could be one particular factor that might be going on. Might also indicate that maybe there's some heterogeneity in the availability or the ability of the workers in terms of taking on more work. Some workers might have the capacity or the ability to expand on the amount of work that they do. And I think this is where again, compared to the first, when we look at the results, this heterogeneity or being able to separate yourself as a more able worker from the other workers seems to be what's at least driving or explaining more of the results. Now, of course, looking at it from the other perspective, of course, the type of advertising that we are talking about here can also move work, attention, work or opportunity from some freelancers to other freelancers in theory, this doesn't seem to at least happen in this particular experiment. And when I think about this, if work needs to be allocated from one worker to the other, this essentially needs to happen, it must be the case that there is potentially more demand than the supply of labor. So, and I try to square these two things together. If you do not have an environment where some workers are losing on work, at the same time, you're not necessarily having adverse selection. How does the supply and balance demand balance look like in this particular market? Because it seems like these findings are going in opposite directions. And theoretically, it's hard to square these two things together. So, a second thing here is, of course, what exactly is happening in terms of when we think about moving work from one type of freelancer to the other, not having the available now sign. What do the buyers think about when they see a sign that says available now versus not? Who is putting this sign? Are the buyers aware that this is actually an advertising that you self-select into, or do they think that this is one of those tags that are put by the platform? What do they think about when they think about the freelancers that they do not see putting this sign up? So, there's also potential this shift or in effect, and I'll get into this a little more, but it might be that when those individuals are not self-selecting into advertising, they're also potentially negatively impacted because of the beliefs of the buyers. So, findings, again, very surprising part to me is that there's almost no downside. The interactions are increasing. The buyers are more likely to reach out to advertisers selling. They're more likely to form contracts. And we don't see a downside effect of this in the sense that we're not taking work from some people and giving it to others. We don't see any potential downsides in the increase. Again, this might be coming from the fact that there's some heterogeneity across workers that they were not able to signal before. Maybe some workers are more able and capable of taking more work. And for whatever reason, before, there were frictions that prevented them from being able to take some additional work, and now they're able to signal their ability to do extra work, and they're able to take on that work. But why couldn't they signal this beforehand? Why were they not able to, especially in the context of freelancing? These are some of the questions that I had, I didn't immediately have an answer to, especially in the context of freelancing, because freelancing is a market where the sellers are reaching out open to vibes, right? They are the ones to put on bid. They are the ones to make proposals. They are the ones to potentially apply to as many jobs as possible. This could be costly upfront. Maybe they are simply reducing the cost of that upfront communication. But again, there are questions as to why can't these workers who are more active not able to signal that they are more active, they are able to do more to the buyers. So, going back again this, I think what's sort of happening at least on first hand is that initially there's a low ability to be able to signal high capacity and low capacity workers. There's low ability for the high capacity workers to signal themselves for whatever reason. And then eventually now we have a system that makes it easier for both parties to be, one party to be able to signal and the other party to be able to see. There's a better separation. And the question that I had was whether this efficiency that we observed, this increased efficiency is really coming from potentially the cut down or that upfront communication and the maybe the unnecessary proposals and communication that happens upfront. The increase don't seem to go down. So, that doesn't seem to necessarily suggest instantly that maybe the sellers are putting up less proposals, but maybe some of that upfront communication is now more targeted towards the project itself, a little more efficient. So, there are these questions about what exactly when we put the system in place, what exactly is it replaced? We are putting the sign up and what exactly does it do? A second thing, again, thinking about this particular context, John also alluded to it at the end. I was again surprised that there's positive selection here. When you look at the advertising and non-advertising sellers, they seem to be different in the sense that the advertising sellers are actually better, unlike what we might expect, except for a few characteristics. They seem to be, for instance, charging a lower hourly fee, which might be against that benchmark. But generally, they are the more positively selected individuals. And again, there's this question as to why is it that these individuals need to advertise? Is it that there's an existing friction that they cannot advertise their capacity to take on more work? It's not only that. It seems like buyers are also more active people. The buyers who are responding to these advertising sellers are actually also, they seem to be different in a number of different characteristics. Now, going back to more targeted comments, I don't find the mechanisms that are at work in this particular example and the findings, in my opinion, are going to be a little difficult. I found the no downsides effect not taking work from one person to give it to another person a little surprising. I found the positive selection, again, thinking of it from a theoretical perspective, surprising. And then more importantly, when I think about as an explanation, I find they heterogeneity that the sellers were unable to signal as an explanation that could potentially put together all the results. But then when we look at the surveys, people do not state the sellers don't seem to state that they actually have different work capacity. So I find it as another surprising thing. Maybe they are unaware, but they are self-selecting and advertising, so they have to be aware. So what might be going on? Is it that multiple treatments are at work? Is it that value change potentially that put up the sign are we changing something else at the same time simultaneously? That is maybe a part for one to grasp? Or is it that maybe some of these groups are really not comparable to begin with it? Maybe it's not as clear an experiment as we think it might be, especially because the control groups are going to be also impacted when we start changing the treatment. One thing also, maybe that no downsides is a surprising statement, it's just a simple statement, maybe negative effects take a little bit longer time to to observe, especially if non-advertisers, non-advertising sellers were positively selected. We might think that because they have the reputation and ongoing work and relationships, maybe seeing a change in their outcomes would take longer. But in this case, since they are not the ones positively selected, that explanation also becomes a little harder to think about. And now we end up with one more question. If this particular mechanism is generating more demand, what creates the credibility that eventually this is going to be information that you can take seriously? I could be putting up signs that says I'm available. If the platform is not monitoring or enforcing that somebody is immediately available, why wouldn't I put this, especially if it's at the right price, because it's going to generate more demand for me. I'm going to receive more bits potentially or more interactions, and then I can choose the one, the work that pays better or is more optimized in my scale. So there's a question about in the long term, what could make, what could create the credibility to this type of advertising that makes it work and still efficient in the long term? Going back to this idea of the buyers and our questions, quick comment. Sure. Yeah, so it's supposed to be a five-minute discussion and we're kind of substantially over that. Just a couple of things, since we had more time, I went over time probably. Are buyers and sellers, is the control group really untreated? I think it's a little hard here because again, as the seller's demand is changing, of course, other buyers demand is changing, and there seems to be at least in the appendix server a few things that hinted that buyers who are self-selecting into responding to advertisers might also be different. So there's that question. I also think when we look into, for instance, these differences, there seems to be a different, even in the control group. I mean, John mentioned this, but this one control doesn't seem to show the positive selection. So I wanted to really see a lot more details about the pre-post differences of these control and treatment groups. I think those are very easy to add. Freelancing is a special context, the fact that you already have upfront communications. What does this particular future change in desperate existing communication? I wanted to see a little more about this, and I think overall, skipping this, I think it's, again, a very great work in terms of trying to help us understand how to overcome the inefficiencies of the labor markets, online labor market, very sort of novel objective. I think the mechanisms right now are, there seems to be a lot that's happening, and I would love to see the next iterations of the paper where these will be a little clearer to me, maybe perhaps. Thank you so much.