 What I want to talk about today is impact evaluators, which are a specific mechanism to try and get communities of agents. These could be people, um, organizations or, um, you know, various groups to collaborate, to maximize some, um, measure towards a goal. So there's a lot of that there, but the point is how do you coordinate large groups to achieve shared goals and impact evaluators provide us a, um, a set of mechanisms to take community oriented funds and allocate them against impact towards, uh, uh, measurable, uh, success toward the goal. Uh, so to set sort of the stage, uh, you have many, uh, conditions across lots of problems where you can break down, uh, the problem space into, uh, the following model where you have a community of agents that are trying to work together against, um, a particular shared objective. Now they might care about this objective in different, um, distribution. So you, so each, uh, so you might have like a distribution of, of interest in, in actually achieving this objective. Um, and you have a different distribution of, um, capabilities to, um, affect, uh, the objective. You also have a dynamic dynamic environment where the capabilities of the agents change over time. So agents might get, uh, more capable or less capable, uh, over time. Uh, in order to, uh, to achieve the objective. Uh, but what you want is to build a system where you can use, uh, shared mechanisms to orient the community of agents to over time achieve the shared objective. Uh, so what impact developers do is that they let you translate a, uh, a measure, uh, and a shared pool of funds into a pretty robust mechanism that can coordinate across many sorts of groups, uh, to orient and incentivize the, the achievement of that objective. Uh, so how are we going to do that? Uh, and, uh, that sounds like pretty basic, but, um, the, uh, and, and impact evaluators are basic. Um, but the key point here is that this very, um, simple mechanism can cause, uh, massive levels of impact. Uh, and, and, uh, to motivate that, uh, the Bitcoin network, uh, and all of the hash rate and all of the consumption of power that the Bitcoin network is causing is given by a single impact evaluator, which is the, uh, the block reward of Bitcoin, rewarding the hash rate contributed on a per block time. So, um, that that's like, uh, the level of power that you can get out of like these very, very, very simple mechanisms. Uh, so, uh, there's impact evaluators aren't like not well, um, understood as, as, uh, as they haven't like really been studied as mechanisms themselves. At least I haven't found, um, uh, a theory of them. Um, there's, they have echoes to, to control theory and to other, other systems. Um, but I've been sort of trying to work on figuring out the components and, and the different like levers that you have to tune the impact evaluator. So, uh, this is like the traditional maybe version of it where you have kind of the Bitcoin block reward or the web three crypto block reward model, where you have some reserve of currency. You have a certain amount of contributions. You have some way of evaluating those contributions and you have some way of rewarding those contributions in the classic Bitcoin example. It would be the reserve is the Bitcoin yet to be minted, um, following some exponential decay. So you have some way of like mapping the reserve to, um, to a unit of time. The contribute, uh, the contributors are all the Bitcoin miners that are contributing hash rate, uh, in a, uh, over time, uh, you can break it up into rounds and think of, uh, contribute, taking the contributors weighing the impact, their impact against, um, against a particular, uh, moment in time. The evaluation function in Bitcoin's case is super simple. Um, and part of why this mechanism I think hasn't been studied well is that it's, um, hidden in the, in, in the model. So, um, in Bitcoin's example, the, uh, hash rate matches maps onto, uh, the evaluation over longer periods of time, not in a single block. So you don't have an actual evaluation function in a per block time, but you have an evaluator that works in expectation. So as miners, uh, compete to mint blocks on expectation, their rewards will match up to the power that they've contributed. And so you only get this evaluation function, um, working over, um, the larger period of time. And so you don't, the, the, this sort of like impact evaluator mechanism was sort of, um, incidental. The, then the reward in Bitcoin's case is just, uh, you, the, the, the coin base minting of, um, the reward to the, to the minor address. Now, that model has been adapted to all kinds of other block chains, right? So, um, Ethereum and many other networks, uh, follow this model. Falcon followed this model. Uh, in Falcon's case we adapted this model to mint not just on an exponential decay, but take into account a, um, a, the reward on a per round basis takes into account a KPI for the network, which is the actual total storage power that the network is contributing. So we're starting to shift the reward structure to cause the impact evaluator to be better, meaning it's not just going to be an, uh, an exponential decay that it's meant to hopefully incentivize that outcome in the longterm, um, mapping the, the value of, of the currency to, to the KPI. But in addition to that, you can measure how well you're doing relative to some longterm goal, um, and reward based on that. So, uh, Falcon uses, um, a kind of this goal against the baseline at the baseline here is a certain amount of storage at a particular moment in time. And so the reward scales relative to how well Falcon is advancing against, against that, that goal. Um, so again, like you, we can redefine the same model in a control theory, uh, uh, sense where you can map to the traditional control theory loop of having some, um, and let me make that and scale that up so everyone can see it. Live slides. Um, so you can have the traditional control theory loop where, um, you have some input signal feeding into a controller. Uh, the controller is providing some system in some input into some system that's going to carry out some action. Uh, that's into the system is going to output the action. At the same time, the system is feeding back into, into some sensor and the sensor is sending a feedback signal to the controller. Right. So, um, again, this is a very simple model, but it, but it's extremely powerful. Tense of the world today operates based on mechanisms, this simple, like many valves, like water valves and pressure valves and, um, you know, systems and planes and so on operate based on these very, very simple structures where you have some way of measuring, um, some state, uh, and you can feedback that measurement of that state into some actuation, um, to control a system. So you can think of any back debilators as that, but for, um, any arbitrary sets of agents in a, in a, in a broader community against some KPI, that's measurable. Again, key here is that that KPI is measurable. Uh, another way of thinking about it is as a, as a valve. So the, so far the impact developers that I described had a reserve of currency. So the Bitcoin example, the FACON example and so on, all have a certain amount of currency that's initialized and then from there, um, it is, uh, uh, minted out and, and reduced. Uh, in Ethereum's case you have additional minting in perpetuity, but you can, you know, like plot out, uh, some, you know, this and moment in time and you can specifically calculate how much, how much ETH that's going to mean. So think of all of those as just having a constant, um, reserve, but you can also think of impact developers as having a dynamic reserve where the, the actual inflow, the, the funds that the impact developer has to offer and to align against the incentive can change with time. So that could change because the impact, um, achieved is feeding back into the evaluator, meaning the impact achieved against some goal is producing some capital return that can feed back into the impact of elevator to reinforce the loop. If you manage to do that, then you can like get these things to like scales much faster than, than Bitcoin. In a sense, kind of Bitcoin had this incidental return, which is the, if Bitcoin become appreciated and has more value over time, then, um, certainly people will, uh, try to compete for it more and you'll, you'll get this effect. Um, but it can be much more powerful. If you can feed back some amount of the, of the value creation back into the, into the, into the evaluator, what this would look like is like closing some fraction of the outflow over time makes its way back into the inflow and you can kind of, um, scale these. Um, that also suggests that you can form networks of these things. Uh, another view into this, like, um, you know, that kind of the diving into how an impact evaluator works, uh, is that, you know, there's a mechanism somewhere in, in, in, um, in a robust way that the community agents can trust. So for our purposes of watching, uh, can do just fine. You have a way to deploy a smart contract, uh, where community agents cannot kind of tamper with the contract. And so everyone can agree on, on doing things, um, and, and working against what the contract defines. Uh, but of course you can create, uh, impact evaluators exist in many other, um, environments. You can think of, um, even like a, uh, uh, we're talking history about, you can think of, uh, conferences in the academic environment as an impact evaluator where like there's like a predetermined number of rounds, like there's, you know, one conference a year or two conferences a year, uh, people submit papers to this, they get weighed, uh, they get sorted. Then academic credit flows back, uh, to all the participants that submitted, uh, papers based on, you know, the, the getting inclusion into the, into the journal or, or awards like best paper award and so on. And so, and that is like a stable impact evaluator that is recurring year over year over year. Um, so well that the community of agents can bet on that impact evaluator continuing to exist. Um, and, and there is one of the key things here, which is this community of agents is able to maximize that KPI and achieve that objective. Um, if this impact evaluator is trusted to be around in the longterm, right? So you want a condition where agents can individually, um, plan out their future and decide whether or not to contribute towards the, uh, the objective and contribute towards maximizing the measure, um, independent of all the other agents. So the ideally you don't, they don't have to carry out any other coordination than just, uh, interacting with the, with the impact evaluator themselves. Um, so in the, uh, in the Bitcoin case or the, you know, the block traditional block reward case, you have some long period of time. You divide it into a set of rounds, um, you know, either, you know, singular blocks, block dimes or something like that. And, um, and at every round, you're going to run a pretty simple algorithm, which is you're going to measure the world, the state of the world, uh, and derive kind of, you know, the, the, you're going to look through, uh, and pick out the signals that you care to, to reward. So you, you, about, you measure, um, how well the, the KPIs being maximized, then you feed that into a weighing to evaluate how much, um, you're going to reward every participant, uh, based on their impact against that KPI. And then you, uh, pay out those rewards, right? So, um, kind of in a, in a simplistic sense, you can boil down like the measure part into extracting a KPI and extracting a set of agents that have worked against that KPI. Here, these are not like single, um, value variables, but, but they're like distributions, right? So think of, um, you know, P here being a distribution of all the agents in the world, um, weighed with their, you know, a measure against the, what they contributed to that KPI. So the KPI here might be, uh, K might be like the total hash rate in a particular moment in time. Um, P might be a distribution of the individual hash rate of each agent contributed against that, against that K. Uh, then, you know, you feed that into, into, um, uh, this kind of evaluation function that weighs that hash rate and maps that onto some, uh, fraction of the, of the funds of the impact evaluator. So, um, this might be, uh, the simplest thing to do is to just, uh, take the reward, assign the reward to, you know, some amount per round and then distribute that evenly, right? So, um, you can, you can think of, um, sorry, I'm getting ahead of myself. That's in the reward function. Um, uh, here in evaluation is just kind of mapping the, the, the impact against the, um, the participants. Uh, think also here is a hook where you can put in humans in the loop. To evaluate really complex things. So for example, there are certain things you can measure, uh, but often translating those measures into, uh, a true proper notion of impact, uh, requires some human judgment, uh, at least today. Um, and it, it can be really useful to, uh, feed those, those measures into a community of judges who can then, um, tune the, the evaluator, right? So for example, um, if you might in the kind of like, uh, conference journal case, you have a standard conference that is happening over time and you have a set of peer reviewers that are evaluating the papers, uh, giving their measures of the papers, doing a very simple evaluation. Like they have to read every paper and decide for, you know, strong accept, weak accept, uh, weak reject, strong reject. Um, and you have a very simple way to like scale this, uh, to be able to deal with, you know, some large number of papers and some much smaller number of, of judges that are going through and you have multiple judges, uh, to kind of, um, get a much better, uh, sample of like what's, what's good. Um, and then from there you can aggregate all those scores and decide what, what you're going to accept. Um, the, the more kind of, um, you could do this for example, for things like events, right? You want to host a crypto Yukon day. Uh, you want to have like many other crypto events around the world. You can measure some rewards, uh, some impact, uh, produce in that event. So it could be like the quality of the talks could be the, the, um, number of participants that attended this event, the ratings that those participants give after an event, like, you know, there's a survey you can like take, uh, input like that. Uh, it could be the long-term views on the recordings of, of this event or something like that. Um, you can, but those KPIs alone often are very gameable. Um, and so you don't want just those measures to, to drive a big impact evaluator. You want to potentially feed that input into judges who are then going to decide the reward distributions. And this is where you can, um, aggregate input, um, and then, uh, structure in a way that where you have limited participation for participation of some, uh, evaluator, uh, judges. Uh, and then of course you like feed that into some reward function. Um, you know, you can split up the fund, you know, in terms of rounds. So this is like the simplest thing to do if you have like some, you know, number of rounds into the future, or you have some other function to scale it like exponential decay or, or whatever else you want. Uh, and then you kind of decide what each participant gets paid out. Now of course this reward function could, could be pretty different, right? Like you can tune, um, as we were saying before, you can tune the reward function based on the success against like API and so on. Um, these, these kind of like simplistic view of like just these parameter spaces is just to give you an idea of how to construct one of these things. Of course you can feed in, you know, the state of the world to each one of these functions and, um, and derive different, different, um, components. Um, and any questions so far? Great. Sounds pretty basic. Raise your hand. Is this sounds pretty basic? Great. This is awesome. Cause that means I'm doing a good job because these things are like, not well understood. Yeah. Like when we introduce this, this judges concept. So this looked like a protocol, but then when we introduce this just concept, it becomes like a DAO kind of structure, right? Yeah. Yeah. I mean the idea here is that you have a protocol that enables participation from some evaluator set that is going to be able to put the, the sensors. Um, so you have these measures, right? And the, the, the, the difference here between this measure, evaluate reward picture is that the, the measure function is extracting some information out of the world. The evaluate function is translating that information into impact against the KPI and then the reward function is translating that impact against the KPI into a payout, into a reward payout. And so the evaluation part in many cases you need some judgment component here that is difficult to automate. And so I'm introducing the judges here because there are a lot of things that are like not easy to evaluate in a smart contract, right? So it's difficult to write smart contract rules that are going to be so good at capturing the quality of some, of some, uh, uh, progress against the KPI that you kind of need judges in the loop to, to be able to do that. Uh, but, but the idea here is like, um, the point is you can use impact evaluators for extremely complex things because you can have like this very intelligent, um, uh, processing in between that can weigh extremely difficult to weigh things. Um, like the conference papers, one is a good example. You have like the entire structure of a journal almost automated. Um, and you call out to like the, the lead people in fields to give their expert opinion on what's good in a field. And that's a fairly difficult thing to automate because that's a highly dynamic thing is the field is constantly changing. You require a lot of nuance in terms of picking what's a good, um, a good contribution or not. Um, and oftentimes even today, like, um, you have like panels of judges that, that, that constantly miss really important things, right? But, um, and so things are working well enough, but like, it's not completely automated, right? It would be difficult to automate that, that process entirely. Like you can do it with just like straight up voting. Like if you, if you, um, had like, I don't know, direct, direct democracy voting on what's a good paper, like you would get like a bunch of error output. Yeah. Uh, so, so the judges model is like a way to get something better than that. Now you could, you could build all their systems that are much more robust than that. Um, but it's just, it just showing it as an example, a way of like escaping out to, um, human judgment. Uh, any other questions? Uh, cool. So, uh, I was mentioning this before, like you have potentially constant, um, reserves or constant, um, uh, plots of, of rewards, or you can make them variable and the idea with us making them variable is that, um, you could have, this might be a way of doing long term public good production when you can only capture the return of the public goods many years down the line. So you could, uh, so for example, you could create an impact evaluator against, um, R and D on something like Filecoin, right? And you, and you, and you want to reward research on, um, cryptographic primitives, research into, um, distributed systems, research into, um, better hardware, um, whatever. You can think of enumerate a ton of things that are important public goods that are going to lead to a better Filecoin network. And, uh, you will only return that value five to 15 years after the initial, um, deployment of reward, right? Or the, the initial funding of some, of some, um, uh, of some work. So you might fund some research. You might reward some research that is produced, but that research has to sift through a big, long R and D pipeline that is going to take many, many years to then translate into some product that is going to improve the network. And so today you don't have like good measures that kind of span this very long period of time. However, with this you, you can do that. You can then, uh, couple of some like return of that result back into, uh, funding back into the impact evaluator. It costs more, more rewards. I'd probably be like better examples on this one. Um, in the event case, imagine that, um, you, you're like rewarding the production of like really great events. Um, and then all the events sign up to, um, uh, give a fraction, you know, a fraction of proceeds over time generated back to the impact evaluator, um, in, in the long, in the long term or something like that. Um, uh, you could also start coupling these impact evaluators into networks. So you can think of, um, uh, these are like not well lined up because the diagram is not perfect, but I need to like draw more arrows, but imagine, um, think, think of each of these impact evaluators as a neuron that is doing some processing on a network where, um, you have some measure of some input impact and some reward output. And so you can start coupling these into more complex networks where the, the, um, impact out of the, the, the outflows of some impact evaluator are going to get fed into the inflows of some other, uh, impact evaluator, right? So if some impact evaluator is succeeding, um, it might reward other, it might be rewarded by another impact evaluator. So, um, you can start kind of, uh, coupling these in interesting ways. Um, you can also deal, uh, so, so far as I've described them, impact evaluators have this problem where, um, they might lead to negative sum games. So for example, if you, um, are going to reward, say, um, bringing a lot of new clients, uh, to the faculty network. So imagine that you want to incentivize a, a large sales team for the faculty network and you create an impact evaluator that rewards, um, uh, closing, um, some, uh, uh, individual, um, client, you might get into this serious sum game where, uh, people enumerate the, the potential clients and they compete really fiercely for the easiest ones to get, um, and they, they don't, uh, spend much, much of attention in, in the other ones. Um, and that is an inefficiency in the market. Uh, so ideally you want to kind of turn this into a positive sum game where you cause these, this like weak coordination to happen in the environment. Um, so, so you don't just lead to the traditional Nash equilibrium that happens just out of participants realizing that they're all going to be competing against each other and so on. You can do something strong, like you can shift equilibrium stronger than that by, um, by making the reward super linear. So you can make the reward itself scale with the impact received. So if you instead change that impact evaluator to say, um, at different levels of different, uh, clients brought into the network, it unlocks different levels of reward. You then immediately cause agents who initially enumerate lists of clients, sign themselves up for different ones, and then help each other to, to close many more. And so you can, um, cause this, uh, this much stronger positive sum, uh, environment. There might be a way to like do Pareto preferred IE. So not just a positive sum game where participants, um, where like the net total is, is, is stronger, but you can actually yield a situation that is, um, Pareto preferred where people are like, um, enter into like this voluntary cooperation spot. Um, and I think a way to do that might be by issuing a rewards based on past participation. So introducing some memory into the system that can track, um, contributions over time, uh, and factor that into the, into the system. Um, and there might be other ways to do this. It's just kind of a speculation. Uh, cool. So I think that's, oh yeah, I had, um, just to kind of, uh, examples again, um, you know, kind of the Bitcoin board reward in the hash rate contributor per block time, per miner. That's like a standard example. The platform block reward is rewarding the, it's measuring the QA power contributed per block time and then is rewarding that with some probability. Uh, you know, you could, you could create like this platform event reward or something like that. Um, great. Um, and that's all I had. Any questions about ideas or can everyone, does everyone feel like capable of going to like produce some of these things and, uh, and, uh, fixing global problems? Uh, yeah. Hey question for you on the, uh, the judges piece, could you just share a perspective on how well built out that spaces? Like as Claros an example, or is there more work to do in the space of judges for these types of solutions? And this specifically in, in kind of the, the blockchain world of like people using judges in, in within blockchains. I think this isn't used enough. Um, I think right now, um, most of these kind of impact about later things deployed in blockchains tend to be very simple smart contracts that are, that are either, um, trying to speculate about some impact. Um, for example, like the Bitcoin case, it's speculating about the amount of hash rate contributed based on how many zeros in a shot to 56 function there, there are, right? And so like, that's like a, a very clever way to construct an impact evaluator. Um, you, but it's coming out of a limitation of not thinking about just other ways of measuring. Um, but you, and that's a very limited thing, right? You can't do that for tons of other things. Um, so both in the measure function and the valued function, I think there's all kinds of ways of, um, collecting information that is not at all explored, right? So, um, another thing you can do here, um, is start to think like, imagine being able to start measuring things about the planet, uh, feeding them into smart contracts, evaluating some impact reward against that and then, and then producing some, uh, some outputs on a classic example here might be, um, you take the Landsat imagery, you, um, evaluate the amount of forest in various countries and like the, the, the planet desire decides to, to reward countries that like increase the area of like forest, right? Like that's something very straightforward that you could do with, with one of these kinds of things. Um, and like that kind of scale output would require feeding the Landsat imagery, running that through object recognition, distinguishing like green tarps from forests. Um, you, and probably enter into this cat and mouse game where like people get more and more sophisticated trees that are plastic to like fool the satellites or the satellite, but the measure function has to get better. Um, and then you, uh, or you, or you can introduce some other model where like you don't just use Landsat, you use like some kind of verifiable claim structure where like you could feed in a, instead, uh, build a network of verifiers that go out and, and, uh, inspect, go on inspect, right? So nation states today use many structures like this to do, do stuff like that. They have like ranges of inspectors, they produce a report, they feed that report into some, some, uh, reward function. Usually the nation state reward function is not positive, it's negative, right? You did something wrong, you get a negative reward, right? So, so they have inspectors, they inspectors measure the world. Um, they evaluate, um, and then you get a negative reward. Uh, but you could, you could create positive rewards too, right? Um, I guess that the other, um, it's not entirely fair to nation states, uh, there are like strong positive rewards in terms of grant funding, like, you know, think of like research funding, um, for, for, uh, uh, long term R&D or, um, carbon credits are a good example where like nation states might buy a whole set of, um, tranches of carbon credits, um, and those are based on kind of, um, that's, that's less impact evaluatory and more impact, uh, certificate oriented. Uh, but you can think of, of that, those rewards as positive rewards that nation states are using today. Uh, yep. When you think about measuring and evaluating those two are very kind of interlinked. Um, do you think about first what can I measure and then I create an evaluation based on that? Or do you, do you think it's better to think about what should I evaluate and then try and find measurements that fit to that evaluation? Yes, your question. And, um, this, this, the, um, separation of measure, evaluate and reward is just conceptual for us to kind of be able to separate these things. Uh, like, like the, already the examples I gave around the super linear reward based on the KPI, like start putting in the measure into the reward or, um, the, uh, um, judges in a sense will end up doing some amount of measurement themselves as well. So think of these conceptual definitions as fuzzy and, and but the key thing is that you need to do some kind of sensing of the environment, evaluating and relating that to impact, um, and then causing a, um, a reward after the fact. Um, I think today like there's a ton of low hanging fruit. You can find what's easy to measure. You can find what's easy to evaluate and, um, you, you have blockchain. So you have like trivial reward, uh, it's very easy to like just, uh, create reward structures based on, on, on cryptocurrency, right? Cryptocurrency account payment is like a super simple thing. Um, so like I would just start by saying, like, what do you care about? Find what's easy to measure and what's easy to evaluate. And then you might find things that are both easy to measure and evaluate. And like, that's a great sweet spot to start. Um, but just because something is hard to measure or hard to evaluate doesn't mean you shouldn't do it because there might be really valuable impact evaluators that you might construct, right? Like the Bitcoin example, um, all, all of that came from was trying to secure the network, um, and trying to cause a delay and a slow down in a consensus protocol to try and like prevent people from, um, and just a way of slowing down block production. Um, and it was achieved by doing this proof of work hash rate thing. And it took like 10 years for people like bouncing around those ideas to eventually land on that. Um, but the idea of coupling the proof of work to a scaled reward on the minting, like that's kind of like what, what caused the Bitcoin success relative to, uh, be money and the other things that can be for, um, like, uh, forgot the, the Chowm, uh, Digi Gold or something like that. Um, so, so there were a bunch of examples like that, but, but you needed like a way of both increasing the number of participants in the network and the amount of security that they contribute and so on. It just kind of got out of hand. And so now we have this rogue impact evaluator that's like, um, burning massive, massive energy and like not doing anything useful with it. So also be aware of that. Like fair warning, be careful what you make and evaluate it because you are going to get maximization of the function. So if your function is not very good, um, you're going to cause like massive networks to, to go in and, and, um, maximize that KPI. So think of like sets of KPIs or, or building valves that like shut them off or something. Yeah, back there. Hello. This may be, this may drive you off topic. And if so, I kind of hope it does. I don't know, uh, funding the commons in your opening speech last time, you talked about, like you just said, I wish more people would pay attention to Dow composability and do stuff. And you didn't say anything else about it. Oh, yeah. Uh, how much does this overlap conceptually to you? I think it does. So if it doesn't, do you want to give me why, why it doesn't then? Well, if dows are like chat groups with a wallet now, when they become something more where they're like, uh, actual API's for them to interface and get real work done between them, then the concept of an impact evaluator seems to matter. Like my, my straw man example is I create us, uh, like a customer support Dow that does customer support for Web three projects. Seems like something you could create an evaluator for. Yeah. And I think especially in, in, um, things like customer support or sales or those kinds of industries sent to have already established very good metrics for what success looks like. And so that's exactly already corporations are using impact evaluators within that structure. And so you can just lift that and turn it into a smart contract. So as a follow up, can you say 30 seconds or less on what you want from Dow Compositability? Yes. So I think, um, there's probably a lot here and it's probably thinking about things that I'm not remembering in the moment, but, um, I think right now Dow tooling gives you this really useful, like you mentioned, it's kind of a chat room. It's not even a chat room. It's like, it's a set of accounts with a shared treasury, right? It's like a very simple primitive. Um, and you can plug in different functions on the decision making so you can have like direct Dow token weighed voting. You can have quadratic voting. You can have many other forms of integrating that thinking. You could, um, you could experiment with much better voting structures, like or decision making structures where you're distilling out knowledge from the participants and using that knowledge to make a decision, which is very different from voting, right? Like voting does not say, um, do what's correct. Voting says do what people prefer, which is not always what's correct. And so if you could like distill out signals and couple that to decision making, that could be interesting. Um, I think Dow's are the Dow tooling represents a missing component that open source projects have needed for a long time, which is a way for open source to have a shared treasury to of resources to then deploy against other things. Um, maybe a way to like deal with impact of elevator is like you could, you could reward long term contribution to a project, right? You don't want to reward immediate random contribution because you're going to incentivize noise, but you could reward long term, um, contribution as evaluated by the participants in the project. Or you might be able to use this. Um, that's interesting. You might be able to use this as a way to solve this kind of principal agent problem with open source projects. So, um, part of the reason why it's very difficult to fund open source is that often the group, the community doing the work on the project has a very strong vision of where the project should go. The funders sometimes like don't know what they want. Um, and they have a gap in, in being able to recognize what's correct to fund. And sometimes you get that flipped where, um, funders and stakeholders have a certain set of goals for the software that are like different from, um, from, from a set of maintainers. And there's a different set of maintainers that like want to take it in a different direction. And so you might be able to couple of these things to orient towards what certain stakeholder communities want out of certain projects, right? So if you wanted to, um, boot a new operating system and you wanted to get a bunch of device drivers to like exist in this operating system, you could like, um, measure the success cases of new device drivers being contributed. Um, and, and how well they work, you need a way of like getting that output and then rewarding all the participants that contributed those over time over many years. Um, uh, so because this gives you away. So if you couple this with the doubt who like this would give you a way of like allowing the measuring and the evaluation piece to factor into the software. Cause the, it's not just about the individual metrics that you have visible in the, in a GitHub repo. Like it's not just the commits. It's not just the issues. It's not just the comments. You want to measure something about the, the output, um, of the software, like how happier people using this, how, how much value is this creating for, for the world in some way. And if you have a way of distilling that, then you could couple like an impact evaluator with like long-term software maintenance. Um, but yeah, there's probably a lot of other ways of, um, using Dow tooling to compose with many other, many other structures. I think like today if there were like much simpler websites to construct dowels and maybe use the Dow tooling without the word Dow, I think the word Dow is starting to scare a lot of people where it just feels like weird and confusing. And a lot of the web to world just doesn't want to touch dowels before that reason. So just call it like a shared wallet. And, and you know, you can open a website and create a shared wallet. And you can then couple that to all kinds of communities. And, um, the function that you decide on how to use the wallet or the, um, the other rights of that, like that well, that shared wallet can first like could be totally composable, right? Like you don't, you don't just want one form of voting, you want many ways of like coupling that. Um, like the creation of corporations is an example of like a shared wallet plus, um, plus some decision-making structure around how to deploy the resources. Um, and, you know, like smart contracts, that was like one or two steps away from creating DAX, like digital autonomous corporations and so on. Um, cool. Any other questions? Yeah, I got one. Yeah, I got the mic. Uh, the question is basically like of what type is world. And, and, and and as some context is like this very similar idea to the PL Journal Club paper that I mentioned to you last week, uh, that we talked about where world was blocks world and it was sort of like a towers of Hanoi type thing, you know, world in the, there was an economy of agents competing to like lift all the blocks, basically exalt the game. But, uh, that was a, you know, 1999 paper and a, you know, trivial example. So do you have any instincts on like what information would need to be in the world struct or something for this to work in the world? Yeah. And in reality, like the measure function is, is really kind of closer to the systems theory version where like you have a sensor like here, the sensor is measuring something about the world. It's just kind of selecting a subset. So you're not going to measure the entire observable universe, but you're going to like you have some, you're going to select the measures that you care about following. But, but the word world here is just kind of meant to encompass like it's sufficient information to you for you to make that this kind of like evaluation judgment. So whatever you decide to measure is what you can evaluate. And what you can measure and can evaluate is what your reward. So if your reward is not giving you good results against your KPI, then you probably need to evaluate better or you need to measure better. What do you think of a candidate of a world like a world where a world is like an IPFS instance and a live P2P instance and like the set of things you can measure out of that. Oh, like, wait, but then it's like, then you can like request anything you want. Yeah. Maybe I think one important thing here is that the for this to work in rounds, you need to be able to take a snapshot about a given moment in time. So you need to bound the how much information is that it's kind of like a world snapshot of that moment in time, ideally including the information that you care about or ideally only the information that you care about. There was a question back there. Thank you. I was wondering if you have any tips around handling the game theory around what to measure, evaluate and reward, because of course just like how rewards can be very helpful. They could also be very destructive if you get the wrong assumption off the bat. So yeah, so the best tip I have is to iterate, like upgrade the mechanism and don't assume that you'll ever get it right. So you want you want structures that can evolve with time and it'll be very contentious because part of the benefit of creating this impact evaluator is to fix the measurement evaluation and reward enough that the community of agents can start betting about the future and making plans individually betting on what reward they can get. So a concrete example of this is Falcon storage providers can because that block reward is there, they can bet on the distribution of reward that they might get and therefore they can bet on facility storage facilities and so on. Same with Bitcoin miners right, they can bet on the hash rate being there so they can bet on getting data centers and getting hash like ASICs and so on and and they can afford to like hire a staff of people that are going to operate these machines and so on. So that entire like planning of a corporation that has to deal with, you know, many months to many years all of that horizon is embedded within the impact evaluator in a sense. And what you get out of it is that like that constancy of that of the thing. However, like you will run into problems like the Bitcoin case is a perfect example. And so you need to make these things upgradable in time to better fit to what the community cares about. And so I think you need if you find robust ways of gleaning kind of the shared preferences of a community and mapping those into the measures and evaluation functions, you want to kind of like maybe check along the way how close are those tied and when you notice a way to like to improve the closeness of those measures then take that step and improve the evaluator. I think other tips like in the meantime, because like that's it's hard to write these impact evaluators with upgradability and whatnot. In the meantime, I would say like bound the return and bound the amount of money that you give an impact evaluator so that like if it's working really well that you can decide to fund it more. And if it's not working really well, it'll like shut off by running out of money. So like if it's it's like being bad, like then you can you like start at every sources and that gets closer to the to the intake valve inflow to add flow model as opposed to the you give it a huge reserve at the beginning. So this kind of what I mean by you can couple them in networks where you could sort of like if value the wiring here is wrong, but you could measure how well an impact evaluator is functioning and use that to like feed more resources into an impact evaluator. OK, so so basically start out with having every impact evaluator be like I guess able to time out after a while by running out of funds and then only continue the ones that seem effective. Yeah, that's like yeah, I would apply evolutionary frameworks to all of these things. You want to try things, see what works, observe the environment and then change. Yeah, like let things die that are not good. I have two questions, but I'll go quick in there. Short one, are there any tools or frameworks for simulating the results of incentive structures and kind of modeling it ahead of time before launching? Not today, because the I mean, the study of these things is so bad that like this is like the you're looking at the the state of the art theory and impact evaluators, which is not very good, right? Like so there are no good. Like there are there are totally many good agent system simulation tools that you can use to explore things like this. So you could take this impact evaluator mechanism is very simple. So you can probably model these in those systems. But that's I don't think anyone's done that work beyond, you know, people, people simulate these things all the time when they are studying some system, right? So if you're simulating and trying to understand how the Bitcoin network would work or the Bitcoin network or whatever, you implicitly are doing these evaluations. But there is no kind of like current software package that tells you like how it, you know, that is very general and tuned to impact evaluators. But that's like a good thing to make. And then a second question is unrelated to this on the slightly higher level. Are there any particular problems like kind of bigger high scale problems that you think not enough people are thinking about or working towards? Sorry. That's a very general question and like my mind like exploded with like a very long list. My design, it's an open question. No, unbound. Complete unbound. Like, yeah, I'm not again. Like, maybe what I'll say is right now, we're in like this really weird moment in history where we have enormous amounts of resources, super good coordination tooling in terms of communication stack, very bad coordination tooling when it comes to decision making, preference, like finding, forecasting, valuing and so on. And so we have this like whole economy that is rewarding things that are not at all well aligned with long term flourishing of humanity or like living things and so on. And so today, like blockchains and crypto and game theory, mechanism design and so on, crypto economics, like the ability to like blend mechanism design with a monetary system and rewards and so on, like very closely and do so in like a world assuming adversarial settings, like the crypto part of crypto economics includes like the cryptography world of groups of agents that are potentially malicious or rational and you have to like you get to kind of build systems like that. You can use all that tooling to then start getting like bringing closer like the reward function with with a measure of like what's good and finding better ways to measure, finding better ways of eliciting preferences, finding better ways of forecasting, finding better ways of like rewarding what's good is like really, really valuable, like extremely high leverage because you can like if you can close those gaps, even like tiny percentages, you know, when you multiply it out by the total like production capacity of humanity like this a lot. So I don't know this is not very specific. Oh, and probably AI alignment that's like another very concrete one that we need more people working that really kind of maybe a little bit of another open ended one is around kind of most of the cases where we've seen people trying to do this have resulted in somebody creating some sort of inflationary token that's sort of a speculative asset to you know, create one of these systems. It seems like the inflationary side of that is really valuable to be able to grow a network and get lots of people in and it gives you more control as a creator. But at the same time, then your reward mechanism, which is supposed to stabilize your network is also speculative and potentially very volatile. Any thoughts on how we can Yeah, that's a good question. So another thing that's going on in in in Bitcoin's case, because it's not just an effective elevator, it's an effective elevator plus their resource that's being contributed is expensive. So in these kind of traditional block reward models, you have a like you have capital inflows transitioning as like inflows into the network through that impact elevator through the block reward. And so they they they're a way of mapping the this sort of like external value into into the network and into the currency without having to go through like just speculative changes or things like that. I think part of the part of that speculation is probably like Bitcoin, for example, like the the you have this the speculation is what's causing the hash rate to increase and then the hash rate increase is then giving some shock absorption to the to the economy. And so the more special speculation there is, the more hash rate will appear and then the more there will be a shock absorber on on that economy, the particular Bitcoin one, because it busts really bad, like the a Bitcoin bust is really damaging, like if a lot of Bitcoin miners are failing out, then it takes a while to like bring them back into the network. So so I think I can't confirm this, because I don't have enough information, but I think that's part of what's going on with the Bitcoin winters, like the halving rate and Bitcoin miners busting and the recovery period of that entire network, I think is what's dragging out the Bitcoin gap. And if you had a much smoother and that's part of the reason why we have like all kinds of systems in Filecoin to like dampen and spread out those effects, like for example, the block reward is like a smooth exponential decay, not like this having like this crazy halving rate every four years, which is like, you know, like you do not design economic systems with that in mind, unless you want a lot of volatility. So that kind of dampening or collateral structures that spread out the impact that can produce like significant short-term absorption to that and then that can stabilize even better. So I think things like that are useful. I mean, beyond that, like there's a separate question, which is so those are like systems within the the crypto economics separate from all that like create utility, like do something that where the impact that you're measuring is valuable, right? So in the in the Filecoin example, the impact they were measuring is capacity contributed to the network and now with Filecoin plus actual deployment towards storing valuable data for long periods of time, right? So and that's what the impact of all the block reward in Filecoin is measuring. It's measuring like bringing in a lot of capacity to the network that you can use. It's not just wasted work. It's capacity you can use. It also brings security to the network and it's saying great, like also use that capacity to store really valuable data. That's like really good impact for what Filecoin wants to be. And so that's an impact value that's much more closely coupled with what the network wants to do. Yeah, I know that that's a good example, but you want to keep going or? Yeah, we can keep going. Is there any more questions? I don't have all night, but all right. Yeah, we're here. So suppose like if we if there's a reward function, so how do we solve this paperclip optimization problem in this case? Oh, yeah, this is like a this is how you make up paperclip optimizers. Like be careful. That I mean judges or like building the traps that we were discussing before like the switch, such as like you let it run out of money or you building some safety switches and so on. Blockchains in general should have safety switches. You have like the really nice model that at the end of the day you can like everyone can just agree to not run it anymore, which is like you need some other system to coordinate that kind of action. But you can do that. So like August model, which I think if enough participants in the network said that a prediction that a prediction bet was an ethical would like shut it off was a very good example of a good mechanism to to ensure safety. Like it's not very sophisticated, but it like works pretty well. Right. Like Oliver has not had, at least my knowledge hasn't had very significant like backfiring of that because there's this inherent problem in prediction markets where like the moment you can take any predictive bet, you can cause all kinds of other bad outcomes out of it. You can like predict what a bad outcome put a lot of money against the opposite bet and then cause it to happen. And so that unethical switch in order helped prevent that problem. But I mean, more broadly, like avoiding paperclip optimizers really means because of what I'm like the paperclip optimizer problem says you have an AGI system that is like controlled by a very simple agent loop that's just maximizing a very simple function. And so that inherent problem like in order to solve it, you have to flip it. You have to like say the the intelligence has to define the optimization function. And so that's like giving the intelligence function the ability to rewrite its own evaluation function. And like that's also scary for other reasons, which is then you have like zero control. So I don't know. It's kind of like an impossible I don't know it's an impossible result, but it's it's like you either have a measure that's really simple. That's like not going to be correct or you let the system derive its own measure that and like just who knows. Hopefully it'll figure out what's good. But it like reduces to like is can you do you get ethics out of understanding the world? And there's some. This is the known as like the orthogonality thesis in in Boston's work, which is like dividing and other people's work, which is like dividing the like your knowledge, like ideally you want the knowledge of the world to increase like the ethical actions. So like the more knowledge you have of the state of the universe, the more ethical your actions become. The problem and so the orthogonality thesis says that there is no so no correlation there, but that's the thesis and it's not proved. And I think like a lot of our future rests on whether or not that's true. So fun times. Yeah, this is taking a very weird direction. Yes. So with the judges part of the evaluate function, there's the human element and judgment may not always be objective. It could be subjective and like different between different people. So how do we how does the impact evaluate or not break? Yeah, I mean, I mean, all measures and all evaluations will be lossy and we'll have some information like you'll create some gap. Ideally, the error rate is very low. So a way to solve that is like by building better better rubrics for judges. This reduces to like all the ways in which judging oriented systems have tried to do this, which is like you build good rubrics for what it means to be a good judge. You build ways of replacing judges. You build ways of like having a larger number of judges that capture more of the space and and so on. Right. So like the judge here is a escape hatch that lets you feed in a lot more complex evaluation work into a very simple mechanism with the hope of like that that is like already better than whatever you were going to be able to write into this function. Right. Judges can very likely be better off than like a very simple thing. Like if you create an impact evaluator and just say it's just going to reward the number of events that you host, then you're going to immediately get a lot of events of like one person hosting thousands of events per day. Right. And like that's not that's not quite capturing what you meant. So you're like, OK, well, at least a certain number of people have to attend. But then just measuring that then it's like, OK, well, you're going to get a lot of events that just have a lot of people coming and going really quickly, but they're not actually creating any value. And so you keep like having to to evolve your notion of what you're measuring and what you're evaluating. And judges are like this like kind of like like ability like the skip hatch that lets you say for whatever metrics you have, feed them to like a very intelligent like a group of very intelligent evaluators that that can like figure out what the preference of the group are and you can use and you can evolve the rubrics in a way. But you know, not perfect by any means and not objective like not only are they going to be subject intersubjective. They won't even necessarily be aligned with the goal set of the community. I mean, there's a whole separate part of this, which is. We don't have good solutions yet for how to take very large communities of agents and get them to agree on preferences like they tend to have like the larger the community you make, the more they disagree about preference sets. Today. A question. So. As I see this a part of the following talk, I was wondering how can you see what metrics in the Filecoin network would be good to measure, evaluate and reward and who would that benefit? Yeah, I mean, there's all kinds of things. I mean, the very basic one to work measures and evaluates and rewards is the bringing of lots of storage, both capacity and actual usage into the network. There's all kinds of other things like retrieving serving data really quickly. The you could evaluate like applications and application development. You have to weigh. But again, you don't just want number of applications. You want to weigh that by some measure of utility of each application. And you could reward the size of the community and the the the the the breadth of the community around the world. You could reward. You could reward R and D in the long term. You could reward like the building of the network itself in very long term times. You could. Yeah, a lot of a lot of these impact evaluators within a network are going to be mapped onto what's good for that network. And so like you could broaden your definition of a network and say, could you create an impact evaluator on Filecoin that rewards something outside of Filecoin? Like and the answer is yes. And we're doing it already with something like Filecoin Green. So that's an example where you're saying, hey, this is like really bad problem on the planet in the world. We're trying to like and and our system is going to cause some impact on that world. Can we both like like undo any damage that we might cause or ideally cause zero damage or can we actually go further and cause a good impact against that measure? So can you like actually start like decarbonizing the planet by having more Filecoin? So like by with Filecoins like ideally correlate the success of Filecoin with like decarbonization, right? And if you can do that, then then you're now having impact outside of the network into a much larger picture and you can definitely do this. But you're now like vaulting. You're using the community resources to impact this broader problem. And so that's something that the community of agents has to agree on. It just so happens that in Filecoin's case, Filecoin is not embedded. Like it's embedded in like a planet where all the community of agents are like going to suffer if like this problem isn't fixed. So like you could argue it's like fundamental to the continuation of Filecoin that like the agents continue. But it's kind of like saying, you know, like modern corporations and capital should like all invest in a reduction of AI risk because that's a way of preserving all shareholder existence, right? And it's like that's not going to fly and they'll record. Yeah, thank you. Although it's so much to try that it'd be interesting. Certainly that's what alphabet should be doing. But you know. So it's interesting that we had a question about subjectivity and then AI because my question combines both of those topics. Great. So I was having a conversation with someone earlier and we were talking about evaluation and I was arguing how I believe that human evaluators are optimal to have in a circumstance like this. I mean evaluating pretty much everything is, I don't know, goodness is subjective. It's it's hard to have computers to make decisions like what is good, what is bad. But they were arguing that they think that with the current state of AI, AI would be, you know, less subjective, more objective about, you know, what is good and should be rewarded versus what is bad as long as you feed at the right parameters around what your goals are. So I was just wondering like with your experience if you agree with the, you know, benefit of having AI evaluators versus human evaluators. So the give like a I won't give a long answer because I would require like writing books. But I will give like a short and maybe like a slightly short, less short answer. So the very short answer is humans and digital computers are like you can have intelligent systems that gather information and make some decision against like a goal. But all kinds of things can go wrong and they have different currently there's different failure modes. So like narrow AI, so like non-AGI AI systems, what they're really good at is processing a set of data and then optimizing against the function that you're giving and then repeatedly continuing to apply that same function, right? So most digital systems that we've made are very basic in terms of following instructions very concretely. So if you can, there's like this program or joke where like if a computer causes an error, it's not because the computer decided to do something wrong. It's like you told it to do the wrong thing. And so it's your job to like figure out how to tell it to do the right thing so that it does it. And so in a sense, it's like how well those AI models are going to evaluate something reduces to how well the optimization function captures what you really want to want to evaluate. And then humans have this other benefit and problem. So the benefit is they have a much broader context of what's good. And so even if you give them a bad objective, a bad function to optimize, like if you give judges a rubric that's pretty bad, the judges will start deviating from the rubric and start optimizing for what they think is good. And sometimes that's good and sometimes that's bad. Sometimes it's really good because you have like thoughtful, well-intentioned judges that are going to produce a better outcome. Sometimes you have bad judges that are going to like decide really like things that are totally uncorrelated with what you wanted them to do. And it's just more correlated with like their interests or some other interests. And so when you have that, like that outcome, humans give you like the full generality of like anything that a human can do, which is like a double-edged sword for sure. And so you have, you can lead to a much more comprehensive measure of what the whole system is able to do because like that evaluate function suddenly goes like, hold on, like, what is this impact evaluate we're doing? Is this good? Like, should we be even doing this? Like all that is available within the that function. And so that's like a super powerful choice. But it also falls into all kind of other biases that we'll creep in. And or not even just biases, like, you could have like rational or malicious actors that are now working against the goal of the system. So in that case, like a very narrow AI system would like perform more predictably, not necessarily better. It's just like more predictable. So it's kind of like a new, like a I said, short. It's like, that was like the less short answer. The longer view is like in the long term, humans are still like processing information and making decisions in the same way that computers process information and make decisions. And so a lot of it go a lot of the distinction is just in how we're measuring how we're processing. And as the capabilities of AI increase, like artificial intelligence is guaranteed or almost guaranteed to encompass everything that humanity can do. And like that gap just means like anything that a human can do and I do, I can do. And so there's like the dumb definition which is like, oh, if you think humans can be good at this or bad at this and like some AI system can do it. It's just not the AI systems that are going to deploy today. The AI systems that are getting deployed today are like really narrow models. Thank you.