 So good evening everyone, it's a pleasure to welcome you to the ninth HMI Data AI and Society Seminar. I'd like to start by acknowledging the traditional custodians of country and pay my respects to the elders, past, present and emerging. So tonight we have a speaker from the Max Planck Institute in Germany who I've just found out is also heading on to the Indian Institute of Technology shortly, Avi Chakraborty and he's going to be talking about incorporating fairness in two-sided online platforms. So, Avi, if you'd like to share your slides and take it away, I'm going to present our work on incorporating fairness in two-sided online platforms. I'm Avi, I'm a post-doctoral researcher at the Max Planck Institute of Software Systems, which is located at Starbrook and in Germany. Now, today algorithms are being used to assist or replace human decision making in several socially relevant domains, such as in banking, to determine whom to give loan, in employment, to filter and rank job applicants, in judiciary, to determine bail or jail decisions, and even in healthcare, to identify patients who need additional long-term care. But as all of you are aware that along with several benefits like scalability and ease of deployment of algorithmic decision making algorithms, there are concerns about biases in algorithmic decisions in several domains, such as in case of employment, Amazon had to scrape the AI recruiting tool it was developing after it showed bias against women. In healthcare, a recent science paper found that the healthcare algorithm which is used across the United States is biased against poorer social groups. And this is one example, I think all of you are familiar with. ProPublica, an investigative journalism organization, they found that a tool called Compass, which is used for recitalism risk predictions in US judiciary, it is biased against African Americans. And the solution what people are trying to propose or come up with is to incorporate fairness in machine learning algorithms. This is in fact a very active area of research as all of you are aware. Almost all the top-tier machine learning conferences these days have several papers on algorithmic fairness. And moreover, several conferences have come up which exclusively focus on research related to fairness in machine learning and other areas in general, like FACT or AIES. However, one point to note here is that existing works mostly focus on supervised classification setting which relies on training data with objective ground rules. Now, what do I mean by this? Let's take the context of creditworthiness prediction. So, essentially the classification algorithm decides whom to give loan or what should be a credit score. And this relies on training data where I mean this training data consists of past decision records where the training data essentially have the information whether some people have returned loan or defaulted on the loan payment. So, these are kind of objective ground truth or some kind of factual information. Although this ground truth can be biased but at least it relies on this objective ground truth to build the classification models. However, if we think beyond classification in multiple learning systems today, the algorithm needs to account for subjective personal preferences. Such as if we think about search ranking, even in a web search, if I am a computer programmer when I am typing for Python, I am looking for Python the computer language whereas a wildlife enthusiast may be interested in Python the snake. So, it's like different people may have different preferences over the kind of search results they are looking for. And the same is true for recommendation as well as when there is a matching system involved. The focus of my research has been to consider fairness in decision-making scenarios where we have to consider preferences of different users. Now, more specifically, I am interested in incorporating fairness in two-sided online platform. And these are the platforms where you can divide the user group into two sides, like on one hand you have producers and on the other hand you have customers. Let's take the example of e-commerce where you have sellers and buyers. In ride-hailing platforms like Uber or Ola, you have drivers on one side and passengers on another side. In content streaming platform like Spotify, YouTube, you have artists and listeners or viewers. And even in case of human research matchmaking platform like LinkedIn or monster.com, you have recruiters and then job seekers. Important thing to note here is that these both sides, producers and customers may have preferences over the other side. Like some producers may have preference for some set of customers, whereas the customers may have preference for some set of producers. So the goal of this research is to fairly consider preferences of one or both sides while we try to develop search recommendation or matching system. So far, we have tried to incorporate fairness in multiple different types of platform algorithm ranging from driver-passenger matching, personalized recommendation, grouping of customers, related item recommendation, ranking recipients in an online charity platform or even location-based search. In this particular talk, however, I'm going to focus on personalized recommendation and either in the Q&A part or even afterwards, I'll be happy to discuss about the other kind of applications we have tried to incorporate fairness in. Now coming to personalized recommendation, these are a critical component of many two-sided platforms where the algorithms suggest relevant items to individual customers, such as which products to buy, which songs to listen to, which movies to watch and so on. Since the item space in this platform is kind of huge, the recommendations effectively determine the exposure for the producers. Like if we consider recommendation algorithm as a black box, this is roughly what goes into the recommendation pipeline. So essentially, when a customer comes, the recommendation algorithm looks at the history. History means like the browsing history or watching history or listening history of this particular customer and also look at all the items which are there in the platform and then recommend K items that would maximize customer satisfaction. Now if we open up the black box a little bit, essentially there are two modules. One is the relevance estimation module and another is once the relevance is estimated, then you pick the most relevant items and then recommend it to the customer. The job of this relevance estimation is to estimate personalized relevance of an item for a customer and there are several techniques to do that. Say one technique is content based where the algorithm looks at the type of items a customer has seen in the past and then tries to match it content-wise with other items which are there in the platform. And there are also another technique called collaborative filtering where not only the history of this individual customer is taken into account, but also the history of other customers and all this information fits into the relevance estimation module and then finally using this relevance course the most relevant items are chosen. The point we are interested in, however, is what is the effect of such customer-centric design on the producers in two-sided platform. Now to answer this we gather two datasets. One is Google Local. So essentially if you open Google Maps you see a lot of local businesses nearby and these local businesses can have different ratings given by the users in fact like any of us can actually go and rate or review a particular local business which is on Google Maps. Now we collected data for about 855 local businesses in Manhattan and the reviews or ratings given by more than 11,000 customers who are based in New York City. We also considered another dataset from last FM so this is a music streaming platform like Spotify and we collected the number of times different artists in the platform were played by different listeners and the data consists of more than 17,000 artists and about 2,000 listeners. And once these datasets are gathered we then apply state-of-the-art personalized recommendation algorithm to recommend top-care producers to every customer. Note that this is not a real setup this is more like a simulated recommendation setup where we are generating the recommendations and then seeing how it would impact both customers and producers. Now when we look at the producer side essentially we see that 20 percent of the businesses in Google Local can be recommended to 80 percent of the customer or like more technically we can say that 20 percent of the businesses get 80 percent of total exposure where exposure is determined by how many customers are business is being recommended to and even in last FM 60 percent artists who are like towards the bottom of the of the plot they get only about 20 percent of total exposure and risk 80 percent exposure is divided across 40 percent of the artist. So as you can see that there is quite high inequality in the amount of exposure different artists or businesses are getting. If everyone got almost equal amount of exposure then the red line will actually coincide with the dotted green line here but we can see in the plot that there is a quite a big gap between the two. Question is why should we care for the producer because someone can say that okay ultimately the recommendation algorithm is trying to do what is best for the customers. So why should we even care to intervene. The fact is that a lot of small businesses and individuals they depend on these online platforms for their livelihood and this dependence has even increased quite a bit after the COVID pandemic hit us because for a lot of customers now the online platforms are the go to platform to do a many of their day to day task and also to purchase or spend money on. And in fact like so at least I don't know the case about the e-commerce scenario in Australia but in India essentially Amazon and Flipkart are the two almost it's like a duopoly between these two big platforms and Indian regulation allows a particular e-commerce platform to only offer a marketplace model. So marketplace means that here the the platform themselves cannot sell products. They just need to be a marketplace which facilitates sale between customers and producers or sellers on the marketplace. However what is happening is that a lot of small sellers in these platforms they regularly come think about being elbowed out in the e-commerce competition by bigger players or bigger sellers. And it's kind of important to note here is often these bigger sellers have some kind of relationship with the marketplace platform itself. So for example sellers like Cloudtail and WS retail they are actually partially owned by Amazon and Flipkart respectively. So there is clearly like a conflict of interest which is going on that Amazon and Flipkart they own these two seller companies and these sellers end up being the most like most sought after seller in the marketplace. Similar concern is also true for hotel business online hotel booking as this hotel aggregator platform called OYO a lot of small hoteliers contain that OYO is biased towards bigger hotel chains and they give most of the sales to these bigger players and in the expense of this smaller hoteliers revenue. Increasingly there is also a push towards going to a fair marketplace. For example like Government of India has recently released a national e-commerce policy which categorically states that the e-commerce platform should be fair to all the sellers. They don't specify how should I mean what this fairness means or what should be the way to operationalize this fairness goal but at least it's kind of clear that they want some kind of fairness in the marketplace. And recently the state of California has passed this assembly bill 5 of gig worker bill which says that the app based platforms need to treat its employees or the gig workers as their employees and immediately as soon as you think about employment you have to give a fair wage and other things. So there is clearly a legal push towards making the marketplace more fairer to the participants of that marketplace or different stakeholders of the marketplace. There are also business requirements to be fair to producers because it would allow a platform to take new producers on board and who may be equally good as more popular ones and it would enable more choice for the customers with higher competition. So the question is what would be a producer centric strategy in this case like if we want to be fair to producers what can be a way to be fair while recommending. One very easy approach which we call as poorest case is to recommend producers who are least exposed to customers in a round-robin fashion. So it means that one customer comes we look up the K list exposed producers so far and then recommend those producers to the customer and this would actually ensure that all the producers get almost equal amount of exposure. So you can possibly see the brown line which is very close to the dotted green line because here it is ensuring that almost all the producers are getting equal exposure. However when we look at the utility of the customers or the satisfaction of the customers the average satisfaction decreases a lot because as you can imagine that now we are not thinking about the preferences of the customers and only recommending based on the producer welfare and the bad thing is different customers can get affected to different degrees like some customers may get recommendation which they like whereas other customers may get recommendations which they absolutely do not like. So the takeaway is that we have to consider fairness for both sides when we think about developing a recommendation system. So towards that we propose a two-sided fairness notion in recommendation where in terms of fairness for producers we propose to ensure a minimum exposure guarantee for every producer. So this notion of fairness is comparable to the fairness of universal basic income that we want to ensure that every producer gets something like no producers to start for exposure there should be getting some opportunity to showcase themselves in front of the customers. And like when we talk about fairness for customers we want the resultant utility loss there bound to be some loss because if we think about topki recommendation that is the best for customers that is what like recommend what a customer is looking for or customer is most satisfied with. Now immediately if we go away from that topki recommendation it may create some utility loss but we want that utility loss to be fairly distributed among the customers. Now the question is in terms of the producer fairness what would be a feasible minimum guarantee and in terms of the customer fairness the question is that what would be a fair distribution. Now to answer these two questions we reimagine fair recommendation as a fair allocation problem. Now allocation especially fairness in allocation has been studied for quite some time in social choice theory where people look at fair allocation of goods. So here you have a set of goods which you want to distribute or divide among a set of agents. The important thing to note here is that different agents may have different valuation over the set of goods and these valuations are personalized. So for example if you consider the red circle or red ball and consider the old man in the top like that man is valuing the ball to be of value four. Let's just imagine that these are an imaginary valuation function but at the same time the lady in the middle values same red ball as seven and the young person in the bottom row like he values the ball as six. So as you can see the same red ball can be perceived as of different value from different agents and the goal and given these valuations the goal is to find a fair allocation but again the question is that what would be fairness in this context. So classically there have been two notions of fairness which has been studied for divisible goods and this problem is also known as cake cutting problem that imagine that you have a cake and different parts of the cake consist of different ingredients. So suppose some part of the cake has chocolate whereas other part of the cake has say vanilla cream and some other part may consist of some other exotic fruits. Now different people may have liking for these different portions of the cake. The question is that how can you cut the cakes such that people are kind of happy. So one notion is called envy-free-ness where every agent values her piece of cake to be at least as valuable as anyone else's piece of cake. So suppose I am standing in a room with n other people I look at my piece of cake and others piece of cake in their hand and I see that okay I mean I got whatever I got is of more or equal value compared to anyone else's piece. Then that is envy-free solution and I am happy about it. There is another notion of fairness and it's called proportional fair share. Now each kind of says that say suppose I as an agent has a total valuation of the cake say suppose I value the cake to be 100 and there are 10 people in the room if I find that my the value of my piece of cake is at least 10 that is like 100 divided by 10 then I am happy. So this is known as proportional fair share that everyone values her share to be at least one by n time of the total value for all goods. However when we think about recommendation these notions may not be exactly achievable or may not be feasible to achieve and there are reasons for it that in case of cake you can cut the cake as much or as many times as possible whereas in case of recommendation you cannot half recommend or quarter recommend. I mean it's like a binary setup either you recommend the product or you do not recommend. So we are into a domain of indivisible goods and for indivisible goods the corollary or the the corresponding fairness notion to envy-free-ness is known as envy-free-ness up to one good. I'll explain what it is and the corresponding notion to proportional fair share is maxim in share. Now for envy-free up to one good is essentially it is similar to envy-free except the fact that we are allowing one item to be there in someone else's bundle for which there can be envy. Say again just coming back to earlier example say I am standing in a room I have got some goods and then I look at some other person and I found that I am okay with almost all the goods the other person has got but there is one item in his or her hand for which I envy him or her. So that is like that is a case where we call that this is a solution which is envy-free up to one good because there is maximum one item for which there is envy. Now if you consider in this example the left hand side box say essentially the agent one has got these three items whereas agent two has got these two items yet for agent one the total value of the items which agent one has got is less than the value of the item agent two has got. Now so in this case there is envy but as soon as you remove the black box then there is no envy so this original allocation is EF1. Now regarding maximum share let's just think about an interesting game say suppose so there is a supervisor and there are please the students now the supervisor brings in some goods and asks one of the students to distribute the goods among themselves or not distribute like partition the goods into say n parts if there are n students and then the supervisor is going to distribute. So note the thing that the person who is in charge of partitioning that person is not going to distribute it's the supervisor who is going to distribute. So then what happens is that the person who is dividing or partitioning the goods in the worst case that person will get the minimum valued partition like that person may be offered the chance to get a partition after everyone else. So if the person is rational the person would like to maximize the value of this minimum valued good and this whole value like where there is a maximization of the minimum valued good this is known as maximum share and now it will be clear like why I talked about this envy finesse up to one good and maximum share just in the next slide. So essentially just coming back to fair recommendation context so essentially in this case the products which we are going to recommend are the indivisible goods the relevant scores the machine learning algorithm has computed are the valuations and the customers whom we want to recommend are the agents and essentially while doing so we want to ensure that for producers like every product we want to guarantee a minimum amount of exposure which is equal to alpha times that MMS that maximum share threshold where alpha can be a value between 0 to 1 and for the customer side we want to ensure that the allocation is n 53 up to one good as we discussed earlier. So in this case say if there are say in products in the platform and there are m customers and every customer should be recommended k products then essentially the exposure guarantee for producers become m cross k by n where m cross k is total available exposure and n are the distinct products so we want to provide every product this MMS threshold amount of export now technically the problem becomes a constant satisfaction problem now forget about the equation in the left side just focus on the text on the right so essentially we want to find product bundles to recommend and there are three constraints one is the exposure guarantee that for every product we want to ensure the MMS exposure threshold then there are EF1 guarantee that for every customer we want to ensure that the allocation is n 53 up to one good and there is an additional cardinality constraint this says that every customer should be recommended k products now in the paper we propose an algorithm fair rate to solve this constraint satisfaction problem now immediately when we look at the effect of fair rate on the recommendations like we then simulate the recommendations with fair rate and we can see that fair rate leads to much lower loss in customer utility compared to what poorest k was given like in this these two plots the green the blue line shows that fair rate almost gives like almost equal to top k recommendation the satisfaction what the customers would get and when we look at the disparity among customer satisfaction it's also quite low because here the standard deviation is close to zero and in terms of the producers it reduces inequality like it ensures that almost all producers are getting similar amount of exposure now this is of course considering alpha is equal to one where we want to ensure everyone MMS exposure but for some platform there can be different values of alpha and then that will determine the inequality among the producers now just going back to the recommendation pipeline essentially we are trying to replace the bottom most module where after relevance estimation we want to deploy fair rate and then give the recommendation and so so the this kind of customization allows us to deploy fair rate pretty easily in already established machine learning models with already established machine learning models for relevance estimation so this is kind of agnostic of the exact model which a particular company has deployed or developed for estimating relevance now just to summarize the key takeaways are that unlike the widely studied classification settings like where like I was following the other talks in the seminar series and most of the people have actually talked about this classification setting where there is one decision maker and there is one like decision taker and essentially this is like a classification decision like either they select the person or reject the person right the setting which I describe and which is true in many other online platform that there are multiple stakeholders who have contrasting preferences like I talked about two sides there can be even more than two sides and when we try to maximize utilities for one side there can be tradeoffs like if we want to make it better or fairer for producers it may actually impact the customers and it may make it unfair for the customer so the important takeaway is that we have to consider notions of two-sided fairness in such settings like where we need to fair to both sides and appear at some kind of a compromise in today's talk we discussed such notions and how to achieve them in personalized recommendation but not only recommendations any information access algorithms in this platform they essentially distribute opportunities right so the key question which we are going after and which I will open to discuss more is how to distribute such kind of opportunities justly among all the stakeholders and with that note thank you and I'll be happy to take any questions which you may have. Thank you so much Abhi that was just beautifully fair and really interesting thank you very much okay so we're going to do the Q&A now I'll field the questions and so folks who are on the on the call here on the panel if you raise your hands as Lashin and Weiwen have done that would be great and then if you're in the in the broader audience if you'd like to write your question down in the Q&A then I will get to it there and also of course as usual we've opened up a Slack channel on our Slack hopefully Michael will post a link in the chat in just a moment for anybody who hasn't joined it yet so Weiwen had a question sort of partway through so I'll start with you Weiwen if you'd like to ask a question. Yeah thanks so actually I have read your triple double paper the fair rack one and what I was doing before is closely related to your work and I have been work with Professor Robin Berg and hey we worked on the multi stakeholder fairness aware multi stakeholder recommender systems and I really like your idea that you consider both producer and the customer side fairness at the same time but I have two questions first one is that your work is based on the assumption that the customer utility or the relevant score is perfectly estimated or accurately given right in real in real world applications um they such value is estimated by a machine learning algorithm say collaborative filtering so it may contain noise and maybe noisy so if the if the utility value is noisy then our evaluation of the customer fairness would also contain noise so what can we how can we deal with that and what if we cannot obtain the accurate utility value. Now this is a great question and yes like you were correct that we didn't consider the fairness in the estimation the relevance estimation module itself right and there can be differences in the amount of accuracy with which you can estimate relevance for different customer so for a and this is kind of more true when you have a disparity in the amount of data for different customers like you may predict it much better if you know more about a customer compared to another we didn't look at that but I have come across some works which specifically focus on collaborative filtering and how can you make collaborative filtering algorithms more fair right and and most of this works what do they do that essentially you have um in the relevance estimation module you essentially tries you try to minimize the the error in prediction based on the past data of that particular customer and in addition to that if you add some additional constraints where you say that oh you don't want to be much like this much oars for one group of customers versus another group of customer those kind of fine tunings are providing some kind of a constrained optimization setup can give you some or can lead to some kind of fairness but yes like this is not solved yet and this is a challenging problem yeah and I know like professor book is also working on some of these algorithms that I have also looked at so possible I'll look up some of your works as well thanks that's really helpful and my second question is that as we all know the main task of main task of recommender system is to recommend the most relevant items to the users so considering fairness will definitely affect user utility or user user preference so what's your opinion on the tradeoff between user utility and producer fairness yes this is interesting because what we did here like that is why like I based on the fact that we are trying to ensure nvp up to one good so we are saying that suppose you have in 10 products to recommend we try to ensure that nine of them are kind of more organic that we try to give you nine items which you like best but there will be one item for which we kind of we try to take away that one item from you and replace it with another item which would ensure fairness for the producer so okay all right and that is why like in the so if you can see the slide here so essentially like we see that this this blue line is kind of close to what you would get with the topk recommendation and the main reason is that like we provide almost similar just beside one item oh okay all right thanks yeah nice questions thanks yes so um the next question was going to come from lexing you also has a comment on this question so lexing if you'd like to ask your your two questions um sure yeah um thanks for the talk um I think the problem setting is very fashion interesting um my question is related to your description just now you say if I'm going to recommend you 10 items and essentially nine of them will be of the highest preference um so can I take this as an interpretation of the underlying algorithm also connection to collaborative filtering algorithms like you see okay my constraint optimization problem actually doing this or there are ways to interpret that so in this case like I didn't exactly go to the collaborative filtering algorithm like so collaborative filtering is used for relevance estimation which we are not touching so we are assuming whatever is being output by collaborative filtering is to be the true relevance scores or this denotes the true preference of the customer but after that I mean unfortunately I didn't have time to go into the actual algorithm like in the algorithm we are like trying to ensure that we recommend most of the things from the most relevant item list and just take one or two items from uh like from some other items where uh like which would allow us to be more fair to the producer so I should just correct a little bit it's not as straightforward as just say taking the top nine items fixed and then like the last one we just try to uh uh manipulate or try to just replace with some other item like there is an involved algorithm goes going on which tries to find the items which can be replaced or which can be put in into the recommendation bundle yeah and in general one thing is this n 53 notion is uh it doesn't actually tell you anything about the absolute value of that particular item or the relevance value of that item it just says that it's like more like a relative term like if you look at another customer and look at the recommendation what that other customer has got you should not be envious of that person except one item so so that allows us some kind of leeway like trying to make sure that we are fair to the producer um so um I thank you for this discussion so is am I correct in saying that the constraint satisfaction problem you presented a few slides ago was essentially running collaborative filtering as a utility estimate right and then instead of taking top k run this right and if you remove the exposure guarantee constraint then it will essentially boil down to taking the top k yes yes yes right because as soon as you remove that exposure guarantee then we will ensure n 53 for everyone and that would be like the best possible in terms of the performance right like the customers will be best satisfied if they get whatever they want essentially the exposure guarantee that constant is uh kind of putting us in a constraint space where we have to find like a solution which also guarantees n 53 up to 100 can I just jump in with a finger on this discussion so um look obviously I'm sure that you could express this constraint satisfaction problem with a a range of different um if you like distributive criteria um sort of distributive principles and I think it'd be really interesting to sort of explore which ones you looked at and um you know how you settled on this one but just specifically to this um point about this sort of a minimum constraint so another general kind of requirement of fairness is if you like that it'd be sensitive to people's efforts um so suppose you're you're one producer and you've done you've worked really hard to make sure that you're going to appeal to the local you know the local community you know you've done loads of stuff through it and then there's another one who's just like I haven't done anything don't really care very lazy they've put their ad up um so and this is the same sort of principle that applies in distributive justice questions um more generally and so I'm curious about um um yeah about why you think that sort of um it's important to give this exposure guarantee that isn't sensitive to um the efforts that are put in by the producers this is great question and actually our algorithm like I discuss it um I didn't want to go into that detail in the top but actually in the algorithm we can tune the parameter alpha like just see that there is in the exposure guarantee we don't say just mms we have a value of alpha either you can keep the alpha same for everyone like for every producer you want to exactly guarantee the same amount of exposure or else you can have this alpha vary based on the effort that the producers are providing so essentially then it would it would lead to higher amount of inequality but that inequality will be justified by this effort based notion right like you can just say that this alpha or the exported you want to provide to different producers are just proportional to the effort they are putting so we have allowed uh I mean at least that is allowed but of course there are other notions of uh like distributive justice which we may not be accounting for and that is why like we uh I actually ended up with this question that the key question is how to distribute these kind of opportunities uh justly and there are a lot of such principles can only be involved and just one point I want to mention here is that oftentimes we computer scientists tend to pick some notion which is easily operationalizable and which can be encoded in a more formal way so that like we can now feed it to like either a constraint optimization solver like we can just take it and put it and give it to a solver or we can try to actually come up with some more efficient algorithms right so and to think about what kind of distributive justice principles in fact can be operationalized and can be mapped to a more formal definition is something which I will be looking forward to so like I'll be happy to continue this discussion even later or even like sometime later but I am kind of curious to know that in which context which kind of principle actually hold like like which holds for a e-commerce platform may not be same or may not hold for say a music streaming platform because the whole context may be very different and we may need to think a little more about those yeah fantastic well this this is definitely the right group of people to be talking to about that yeah so um michael would you like to ask your question yeah thanks and thanks for such a clear and covered so many concepts in this talk it was fantastic I have a thought which is how would you characterize the difference between like this I guess static recommendation system and something that's more like a banded solution to this type of problem and I wonder if like the band solution would maybe somewhat nationally like handle some of the not necessarily but could address some of the exposure problems especially if like you just earlier some of the problems are due to the high uncertainty for certain like relevance estimates right a banded framework make that explicit and then like try to target that yeah yes yes yes that is correct in fact like we are now trying to work on a banded setup with the same kind of goal like how can you ensure that every producer or effectively it's like every product is being part of that exploration phase at least some minimum amount of time and just guaranteeing that would actually ensure that we provide some guaranteed exposure for everyone so we are looking at that and that is an important point you make and that might help us also circumvent some of the problems with the collaborative filtering methodology that's correct thanks michael I'm going to come back to distributive justice in a minute but um leshing if you wanted to ask your other question oh um okay so I'm just going to ask about the algorithm a bit more um so you said there is a polynomial time algorithm to solve this allocation problem is um is that the optimal and right where's the limit in these like a social allocation algorithm but I I admit that I'm not familiar with them so does it find a good solution does you find the best solution okay so um let me just just give me one minute I'll I'll just pull up the slides for the algorithms I think and it should just help me to answer your question so so essentially uh can can you see the slide on this okay we're still on the same window as before uh can you see now okay we're on ensuring two sided fairness now yes yes yes so so essentially this is the algorithm like fair rec is uh so it's a two phase algorithm like in the first phase we want to ensure the guaranteed exposure for producers while maintaining f1 for customers and then in the next phase we continue allocation until all the customers get exactly k products let me just give you one example like how it works so suppose we have to recommend five products uh there are five products to seven and seven customers and every customer should be recommended three products and the exposure guarantee we provide is say every product should be recommended to two customer now what we do is just we take this number two like what is this exposure guarantee which creates two copies of each product and then allocate all these products to the customers now the question is that how should we ensure if one for customers we essentially allocate in a greedy round robin manner that suppose we look at the the preference of the say say we order the customers it's like an arbitrary order but we pick the first customer and allocate the product which this customer likes most then we go to the next customer we again allocate the product which this customer likes most but then if this we continue doing that but at some point we may find the situation where we allocate we try to allocate a product which is not available like we have already allocated to enough customers so then we go to its next preference and that is how the whole algorithm kind of proceeds so essentially we keep in mind these two constraints one is availability constraint another is feasibility concern that one customer cannot be allocated the same product more than once i mean here when i am talking about allocated is like recommended now say suppose this first phase ends and 10 products are allocated to seven customers and greedy round robin leaves each customer with either one product or two products now we still have to allocate until everyone gets three products so in face to what we do is we make every product infinitely available and this can include the products which earlier we couldn't allocate and then we continue doing this greedy round robin and the interesting thing is that this particular phase actually allows us to make sure that the utility or the satisfaction of the customers is as close as the top k thing as possible so essentially this is the kind of algorithm and in terms of the theoretical guarantees we ensure cf1 fairness for all the customers and when let's take the value of alpha to be one we ensure mms exposure for at least n minus k producer where n is the number of producers and k is the recommendation size like say suppose there are if there are 100 producers and 10 icons to recommend then essentially we'll ensure that we provide mms exposure for at least 90 producers but in practice like n is much larger than k so we almost cover everyone and the orskies runtime bound is ordered of m cross n cross key like which might be large like to look at but it turns out that we can run this in batches like maybe the platform may run it every five minute window and just consider all the customers which have arrived in that five minute so that way we can make the value of m lower k is anywhere low so it only depends on the number of items in the marketplace. That was really fascinating so look i have a few additional questions i don't know if there'll be time for them but i can put some into the slack afterwards but i was thinking about so in any in any sort of problem of distributive justice there's this question of like you know what's the thing that you're distributing and is the thing that you're distributing the thing that really matters right so if you if you equally distribute resources like money say to a population and the population has different needs like someone needs to have you know extensive modifications to their home in order to accommodate their disability then you're you're not going to be getting equality of outcome even though you're giving everyone the same kind of resources so i was thinking about your the two things that you're distributing on the one hand there's exposures and on the other hand there's utility and so i wanted to just sort of press and think a bit about whether those are the things that really matter so for example if i'm a if i'm a business in this situation i guess you know i want exposures but i only want them as a means to an end and what i what i really want is exposure to customers who are likely to click so if i'm getting the sort of um you know the the the the guarantee fulfilling kind of token exposure to someone who's very unlikely to click on me um then that might it might still be the case that the that fair rec um leads to some cuts some producers getting way better actual sales like the outcomes that they really care about um and than others um because of the way the algorithm is working um so yes how do you how do you think about that that that is correct so one thing which we have um so again just coming back to this thing that ultimately we are talking about recommendation in this case right now either we can view this as an allocation problem that is essentially as if you are allocating the products to customers or else we can view this as a matching problem where you essentially match producers with customers now in case of matching problem you tend to deal with preferences from both sides like in this case we are kind of assuming that producers are happy as long as they're being exposed so there is no absolute preference over the customer but we can bring in this aspect that the producers may like you can have the customers and the you can know the likelihood of a particular customer clicking on a product and based on that you can compute the preference on the producer side and of course the relevance estimation model is going to give you the preference on the customer side and then basically you need to think about how to match and try to be fair in the matching process so it's just like technically it maps to another problem but of course conceptually we have to think more about exactly what opportunity you want to distribute like is it just about exposure or it's more like the probability of sales yeah that's fantastic um look i would love to continue this conversation um and it's six o'clock here so i think folks may be a bit slow to get on the slack and might pick it up tomorrow um because i'm sure everyone's as hungry for that dinner as i am um but thank you so much abby for for the talk it was really great and i hope we'll continue um to keep this conversation going over the next uh weeks and months