 air conditioning in your car, whether it's recirculator or not, and whether people should like if you're getting sleepy in your car and you've got on air, you know, maybe it's a CO2 level in your car. Totally happens to me. Yeah, I can't be circling on for like more than five minutes. Yeah. So it's not just for COVID. When people talk about air quality, it's not just for COVID or other viruses. There are reasons why you want, I mean, there are reasons why air quality is important. And there's, and you can't measure if there's COVID there. The other way you could tell is if there's good, right now we're using basically CO2 meters, monitors to see whether or not how the CO2 and if the CO2 is hot. That means you're recirculating the same air, you're not getting. But yeah, there's a lot of different reasons for having indoor air quality standards for a different reason. And you don't even think about like buses in your bus driver or your car. So I want all of us to give a round of applause to Joanne and Peter for good homework. Wow. So, as promised, we have it set up like a Sudoku. Sudoku. Are you intentionally sharing just like a partial of your screen? It looks like we're only seeing the left column of, there you go. I think Peter yours is more up to date than mine, so we'll use yours. And if I can actually step in for a second. I missed all this activity happening so can you catch everybody up to what happened here because I knew something might have happened but I didn't realize it was going to go this far. So I would love to have some way to bring everybody up to date who listens to the recording. Oh, good thought. So we decided last week that we would work on a topic per week. And it could be a divergent meeting, meaning we just figure out what the question is, or the topic and the question, the solution that we're trying to make sense of, or actually have a question to make sense of, and then it was a tool, in this case policy keys to parse out the solution and give it a non partisan rating. So, the process is pick a topic and that was indoor air quality. We had a question should there be indoor air quality standards. And then during the week. It was obvious that there were two choices. One is impose those standards on everyone in the US, or just impose it on large employers. There's no doubt that large employers would score higher than imposing it on the entire US. And there's lots of analysis about. I think it's called over stuffing bias. You know, so try to make in policy keys world we try to find a solution that will solve 80% of the problem. Not not the whole problem. Once you solve 80% then what's left over is a new problem to solve and then solve that separately. All in one, and you lose a lot of support. So, the process here is then to get 16 yeses, or you might call them pros. I just don't like pro I don't like pro and con, I prefer yes and no. So 16 yeses, like a Bayesian sort. The best ones, the best 16 out of however many you've come up with. And then the same thing on the nose, Bayesian sort the best 16 you can come up. Roughly speaking the colors. There's a little key down here. I think solutions come in four flavors. So emotion. And maybe the M stands for money to the momentum is probably more correct. I for interesting information. And then T for time span you notice it spells emit. And well as well as time going this way and admit going this way. Amit was chosen on purpose because in decision making bias and noise our sisters. We hear tons and tons of stuff about bias, but we don't hear very much about noise. Until Daniel Kahneman wrote a book about it a couple years ago with diversity. And a few others. So noise is when you simply have a hard time parsing through the information. So what we're trying to do here is parse through the information to get a clear signal, the signal from the noise. So roughly speaking in the puzzle and this is not exact. We, we try to sort the reasons. Like for emotion yes and for emotions no. For momentum yes for momentum no. For interesting bits of information yes for no. And for time spans yes and for time spans. Why it just makes for a better narrative. And a previous actually, we have. We have this in a dynamic form. And actually we can, we can push a button. Well, not that easy, but pretty close and get a 1500 word report. That would be for people who prefer paragraphs to charts. Okay. So, Jerry, is that should I keep going or is that enough of a pre. That's a great start. Thank you. Thank you. Thank you. Thank you. John and Pete, if you want to step in and talk about your piece of it, you've got your hand up, so please jump in and then, and then when we go back to you, John, I am unable intuitively to read the policy keys output here. I see consultants with 23 in the middle. And I just, my brain does not know what to make of it. If you could talk us through the read after. So I feel like what Peter and I did was knowing we were going to, we, John set up a time for us to meet. So Peter and I, the night before we came up together with, it was over 16. We had like maybe 20 or a little over 20. Yes and no's for, and at the time we were working with, should we have indoor air quality standards? Should we mandate indoors? And so then we met with, he, he took it and did a, a first rough graph draft of this. And after talking about it, the subsequent stuff that's when John, we named it to large employer. Like he explained that why he wrote to large employers, keep indoor air quality standards high. And he's been. Improving improving the thing. So Peter and I don't have anything to do with what this, look, this is John's. Layout. Layout. So we were kind of subject matter experts around air quality. And John's. John's come. John's the policy keys expert. Yes. And Peter and I aren't even quite sure how this works. After our talk with John, Peter said, I still wish we could have a run for it. Peter and I were able to contribute and then have a long talk with him about pros and cons of indoor air quality. And we gave them some references. So we could have a run through of how this works. The puzzle. Me too. Yes. Okay. So. Let me. All right. So we've done the outsides. It's like the grocery store. We've done the outsides, the fresh food. Now let's go to the, the aisles with the package food. So. There's 128 roles. And this is painstakingly built. Are these 128 roles consistent from puzzle to puzzle? Yes. Okay. So these are, these are your 128 normal roles. Think of this as the scrabble board. Okay. Or the chess board. I mean, it's set. Will there be edits to it every couple of years? Sure. But not now and not for a couple of years. This has been highly vetted. This has taken two and a half years to put together. And it's, it's solid. It's holding. It's, it's working. Remember, there's AI attached to this too. So. You move two pieces. Oh my God. It would be like three months of coding. You know, so. Because I'm not a good coder. Maybe somebody else could do it faster, but I can't. So this is like, we're locked, locked and loaded here. All right. So Jerry, you said consultants. So let's just give up. This is. The abundance governance side of the table. So this is special interest groups, Republicans, Democrats and independence. This is the abundance commerce side of the table. Where are we supposed to know that that's abundance and governance? How does, how does that even show up? Well. There's a, there's a bar down here. Wow. Okay. I would know that. I would never have found that or understood the correlation. Okay. Well, yeah. Bear in mind that this also is attached to a website. That has more information on it. Yeah, except. Like it is nowhere obvious that that first row of four rows is that thing. You have so much information packed it. This, this reminds me of Edward Tufty. Who is a really good. Chart designer and a terrible UI designer. You don't want Edward Tufty ever doing UI. And I'm, I'm not trying to insult you when, and your UI design here. I'm just trying to say that there's so much here to unpack. That's not obvious. That I'm thrilled that you're talking us through it. Okay. Thank you. So. Okay. Let me start. Let me back up for a second. Okay. AI loves ground truth. If you can find that, ground truth in AI, that means that you can add all sorts of mathematical wonders to it. Because it fits into the mathematical world. So the ground truth is. There's a time to save and a time to spend. The time for freedom and a time for laws. Where can we agree? Nice tidy four square. And, and sorry. We're, we're not going to be able to do that. We're not going to be able to do that. We're not going to be able to do that. We're going to be able to do that. We're not going to be able to do that. So the four square. And sorry, where, where is that statement? Or how does that relate to? Well, that's not, that's on my main website. Policy keys. So now we have a ground truth and we have a force where. We can do weighted averages. Oh my God. The wonder of weighted averages. So if you're a mathematician, that's a wonderful thing. All right. So the four sides of the table, then. So, governance, thrift. And commerce. When you make pairs of them, like DNA, you get abundance governance or abundance commerce, thrift governance or a thrift commerce. So that's what these four sides of the table are. That's abundance commerce. Sorry. Abundance governance. Abundance. Commerce. Thrift governance. And thrift commerce. So. Abundance governance is large governments and NGOs. Abundance commerce is new businesses and technology. Thrift governance is local governments, guilds and consumers. And. Thrift commerce is established supply chains and jobs. And again, this AGCT format is fixed across all puzzles in policy keys. This is another aspect of your model that. Drops in each time. Yes. Okay. Thank you. Then on this column, this is a very strong bias for change. A little less strong. A little less strong. And then this is. A slight bias for status quo. To a large bias for status quo. So this is a level playing field. And that's the lower axis of the matrix that has ABC change bias status quo. You're describing that axis along the columns. Right. Cool. So that gives us. The four sides of the table. Now let's do the walls. I call this a wall. Eight of them. Funny thing. Each wall has their own media outlets. So we search those. We try to make sure we don't not include. Media and think tanks from each of the eight walls of information. Because otherwise we're just spinning. It's just a fancy propaganda called journalism. No, this actually is journalism. We're trying to balance all the eight walls of information. We're not saying, oh, you're bad. We're not going to listen to that. So we're trying to balance all the eight walls of information. We're trying to balance all the eight walls of information. So actually there's every, every wall has good points. It's up to us to find the good points. And then I call. Half a wall a window. And. It's kind of like subcultures. And you can kind of see that there's kind of a little subculture in each one of those windows. And then there's the party leadership. The doves in the Hawks. And independence that are spread over justice. And deep reading. Who are the kinds that can actually be influencers. Kind of are their own little subculture. So John, it's, it's, it's, it's, I, I'm sorry. I'm going to keep interrupting unless other people don't want me to wrap, but I mean, I'm trying to clarify as we go, because I'm going to forget all these places where I'm, I'm losing traction. So it dawns on me at this very moment that the arrangement of all these subgroups of the 128 subgroups into this matrix is consistent from puzzle to puzzle. It's only the numbers that are changing ever. This is the matrix. Yes. Shit. I did not realize that I had no knowledge of that. It is not obvious. I thought they were moving around according to some poll you were running or whatever else. They're not in motion ever until we go to some interactive form of this, which you haven't built yet. This is the game board. Just like chess, checkers, Scrabble. It's the chest and checkers. The pieces all move around these pieces. These are the pieces, but these are, these pieces are not moving on this board. The numbers are the, the numbers are the pieces. Got it. The numbers are the squares. So it's, it's, it's not even like a, like an arrivals and departures board in the airport where everything moves up and down. Okay. So we're going to wrap this up. Yeah. The first one is the matrix. Sorry. That was unclear to me before. No, thank you for asking. So the idea then is that you want to put the pieces number two. In any of the places of which, well, I'm getting ahead of myself. The first cut is easier. The first cut is, is, let's take one. That I've already scored sciences. So. or not in favor of keeping indoor air quality standards high for large employers. I think it's fairly obvious, so I scored it yes. Now, the question is, is of all the yes reasons, which yes reason makes the most sense. And after doing a Bayesian sort, I came up with preparing now to slow the next pandemic. So that's what science. This is your subjective judgment or this is AI and algorithms helping you choose. Yes, both. It's condiment in his book noise suggests that to reduce noise. You want to start with a human, then do a big mechanical process, and then end with a human. So what happens is that you get fairly obvious answers out of the AI. But let's take an example radiology for x-rays. There is spectacularly good AI for reading x-rays. Is it 100% right? No. It's probably in the mid 90s. Is a human better? Sometimes, but not often. But that's where we are right now with the world. The machine can do it probably better until it makes a mistake and it needs a human to find out that that I've seen that shadow before and that really is cancer. Let's do a biops. Or no, I've seen that shadow 100 times before it's not cancer. So, Joanne, you have your hand up. The question is, you said, just the science have an interest in keeping air into our qualities high. Do we have, I thought we, this was talking about whether we should create new into our college or do we already have high in our air quality when it comes to things like COVID or I would call that being a little too deep into the weeds. Okay, this is for sentiment and direction. We're not actually writing the law. Scientists will pick the number. We just want to like politics, politics is basically sentiment and direction. Because if you're saying keeping that means it already exists and I think you'd have more people saying yeah let's keep it but if you say, are we making a new law then you'd have fewer people on board with that. That could be the wrong keep could be the wrong word. Yeah, that that was just how it hit me but other people see it differently. Real quick, Matthew is wondering what the numbers meant to. Yeah, so science. Let's go over what I said so the AI and I will said it was fairly obvious that science scientists would be for indoor air quality standards and large employers. Again, we might need a better word than keep. And so that's a yes. And so we didn't shade it shading is no. And then, once we have it as a yes. We do a Bayesian sort on which of these yes reasons, most highly correlates with the science scientists saying yes, and these are the numbers. Okay, so yeses are odd numbers and knows are even numbers. I think for clarity, the numbers that are in each of the cells are not are any kind of rank order they're merely pointing to which of the issues that's either yes or the no column is the strongest correlate to that cell. Yes. Okay, that was also not obvious to me whatsoever. Yeah. I thought about letters. But there's 26 letters and 32 reasons that didn't fit well. I'm open. And the other thing is number words, letters and numbers kind of look right to me in the situation, whereas letters and letters was kind of like weird. So I have if I could find something better than numbers I would use it but right now I don't have anything better than numbers. I feel like maybe if we're trying to like talk about optimal interfaces. I think this is fine personally but like if we wanted to see other things quickly rather than just the reasons then I could see like it with software just like reaching to a different view that's like the same puzzle but then like maybe instead of numbers you just have like a color that's like red for red or red for no and green for yes or like depending on what you want to convey. I could see different, like especially when you have so many like groups 128 roles, just like showing one thing at a time. I think is ideal so like for getting the reasons I think this is. I can't really think of an obvious way to convey all these roles reasons with more than just a number. Thank you and I think you're right to because different views so we could hit one view that would be all the emotional momentum. Interesting points and stuff and have that be a color spread on the heat map. We could do. We could take the numbers out and just have the, I think Jerry said this last week, just the yes and the no on a heat map. So that yeah that could be tons of new views. Remember to know this is this is kind of like behind. This is kind of the back of the house this form. The front of the house is the game where you're only doing eight roles at a time. You can stop whenever you want. So this is you're actually seeing like the guts of the AI, you know, will not quite the guts you're seeing the step before the guts of the AI. Just to clarify before Jerry gets in there sorry like that that the normal puzzles are like the same look as this except less than just the eight roles or however many or 16. If you go to policy keys.com you'll see the game for this week I posted it this morning. And that's the format that people play. Right. This is the format that is a one page narrative tool. Okay, Jerry. Thanks, and when when a player comes in to see the eight roles you've chosen those roles are chosen at random. But I could have and we might still do that in the future but right now we call them opposites of the game board. So in other words, I forget exactly the scheme I use but it would be something like activists and sanitation, gun owners and caregivers, big tech and urban investors and national lenders and urban part time. You want that. Some I'm sure some of you know about doors. Kerns, Godwin. Good one. Nice work. I didn't quite get. I don't know if I got that quite right still we knew who you mean the president. Thank you and she wrote a book about Lincoln. I think it was called teams of rivals. Yes team of rivals. We designed the game around that concept. We want, we didn't want to have gun owners and the corporate lobbying hawks and border and order and one game board. We wanted to have opposites and see where where like oh my God they actually can agree. Well that's the point. So two questions then. Are you always picking from the extreme left and right columns for those opposites. No, it's sequential. So it just it just, it goes to 16 pairs until you get to the center. Gotcha. And then the other thing I was thinking was what's hard to see here is that 27 shows up a few times like which are the popular numbers in the hits is an interesting data. Yes, that's a very interesting question and that has come up before and so here's the best way I can explain it so far of the 16 yes reasons and the 16 no reasons. There are two types. I already said flavors for the other thing so I'll say two types. One is primary reasons. And the other one is explanatory reasons. I don't know if it's going to be a decision, maybe not something someone would make their decision based on but really interesting to know, kind of like maybe number 12, and number 11. I don't know that anyone's going to score 12 or 11 as the reason that a role would go. I'm a I'm for this or I'm against this, but it's awfully interesting. I mean, I had no idea what Merv meant, and Peter took me to school. That 13 will actually get COVID it'll trap it in there. But if you're really, really scared, you might want to get a HEPA filter. That gets almost everything. Now, on the other hand, just like cars, when you can crash and not die, they would cost a lot. Whereas cars that you can afford to drive that you might get hurt in a crash, they don't cost as much. Right, so now we're thinking, as you can see there's a sort of a probability thinking here, not, not totally binary. Other question before we actually play the game. Anyone else other questions I have a comment if nobody else has a question. There being nobody else raising their hands. So this, this is just a comment on today's session and what we're doing right now. It feels, it feels like this is sort of a feedback session on policy keys and a test drive of policy keys, more than sense doing into air quality and air filtration. And if you would like another session where we do it the different way without policy keys and go through the issues, you are welcome to do a rerun or whatever because because I part of me really wants to map the air filter issue in some in some deepish way, which I don't think we'll get to do on this call which is fine with me on this call entirely. But for the purposes of sense doing where we came to last week was or maybe the week before was suggestion that we could, instead of having what john's called the virgin meetings we could have, we could have somebody say hey, I've got a tool. Let's bring a tool in and talk about it. And so I think we're accomplishing what we set out to do. And I hope that we have some time today for us to interactively improve what john's got here, you know, to learn, learn, you know, he's got it, he's got a few things filled in and hopefully we can do a few more together as a team. I have both a kind of I have a love hate relationship with the whole concept of air quality. It is crazy making to think about, you know, the pros and cons of it. It's like it's an obvious thing to me. So I personally don't need a lot of sense making done around air quality, or I'm not super interested in in doing that, even though I think it's incredibly important. So, so thanks, Jerry. But I think we're doing what we're doing. But I think that maybe Jerry was saying that. So, we don't realize, since we read about this stuff all the time we don't realize that other people don't know as much and I think for sense making. I think it might be worthwhile, because air quality is such an important thing. It's something that our government and other government should be talking about or even if you own a building or if you own a business or if you, you know, like, like we've gone out over the weekend if you drive a bus or take a bus you know you care about the air quality. So, I'd be in for doing another one since just in, like Jerry said, since making your phone. Yeah, because it is an important topic. I feel like with the 16 yeses and the 16 noes we've sort of bypassed the discussion about what this all means. Yeah, and I was looking forward to that discussion. Now, that's an interesting point. When you try to score a role, yes or no, and why what I found after doing almost 100 of these puzzles now is that it makes me get to the sense of the matter. Because I'll realize that three or four of the nose aren't really that important, but for a number of the roles on the board there's something else that is. You can come up with a group of like minded people, because like minded people will score group think. Right. The only way you can actually get to get a full spectrum of making sense of something is to think about how a diverse group of people would think about it. Now, this board is diverse people. It is painstakingly a model of the United States. It is not a painstakingly good model of Spain, it would have to be modified or California, it would have to be modified. This is a board for the United States. Now, it would be hard to say that there's a bias here, because it's, it's spread out over 128 roles. You're going to have a bias which be jumping out at you right now. And I don't know how it jumps out because we got consumers we got jobs. We got government we got commerce, we got spending we got not spend Jerry. Now you're muted. Thank you very much. Thank you for doing this so often I just wanted to put something else in the conversation, which is that a virtue of your approach john is what Adam Grant calls complexification in think again. And he basically says that when when things look binary they're often not binary there's a grayscale and there's a whole series of layers to the issue. So if you take them apart, you can find all these interesting new combinations which I see that you're doing very explicitly here. I love that book. And I like that a lot. And then I'll give a one second Tracy of in my management 101 class and business school, we did the ugly orange case he divide and I'm going to give away the plot. He divides the class in half gives everybody a case that says there's a shipment of the last ugly oranges in the world coming into port you have a million dollars to buy them. You must have the peel from the onions because you're going to create a serum that will solve cancer. And it turns out that the other half of the room has exactly the same thing except they need the juice of the oranges. So if you can, complexity issue and talk to the other side with in a non adversarial way you can give the ship owner a buck for the ship. And then each of you can go do research and solve cancer and do whatever else. It's a very interesting case because it's like, Ah, okay we don't always want or need exactly the same thing. And so, getting people in that mindset is really, really interesting. I use that exact I use that exact case study when I was building this. Oh, seriously. I seriously did. I love that in my Vistage group we had a speaker do that orange thing like 10 years ago. I love that of like 20 business owners around the room. Three got it. And 17 didn't get it. It was it was a negotiating workshop. Yeah. So the moral of the story is, if you don't talk to the other side and find out why they really want something. You can't do a deal. Well, you can do a deal will be a lousy deal. If you talk to them find what they need, you can do a good deal. So here's the same thing with Republicans and Democrats. I kind of, okay, or let's say change people in status quo people that's better. If you find out what change people says cool people can agree on. You can do one. Super cool. Yeah. These proceeds. Okay. So, the idea was to play the game today and so we've had a half an hour preamble. So what we could do is, if you want to read, well let's just do one. So, we'll do, we'll do what we did one that was a yes let's pick one that's a no. So the AI and I decided that it was a fairly obvious answer that the corporate lobby would be like, all against this. Okay, so why, what would be the, what would be the best reason of these. And we came up with yet more government regulation. I mean, oh my God, have you seen was it like a million and a half pages of laws. You know, I mean there's nobody can actually read all the laws in the country. So it's like no more. Okay, now you may disagree with that maybe you think it's You know, we already have good air into air quality we're the top 10 nation. Well, that's fine. Okay, we, but it's the, yes, the reason isn't as important as the up and down call, because that's where the AI does a really great job. The reason is, okay, more, more neuroscience. Arielli, I think, Dan Arielli. You walk up to somebody in line and say, can I cut in front of you. They say no. A certain percentage. Person cut does the same thing again and says kind of cut in front of you. My child needs to get their lunch. And I'm going to be late. And obviously way more people. Let the person cut the line. If there was a because statement. The numbers in the game is the because statement. So I used a reality's research to build this. In other words, if I say corporate law is against this, you might go, I don't know. I mean, I think, I think they might be for it. But if I say, okay, well, it's because they don't want any more government regulation, because there's a million pages of regulation already, and it's too much. And businesses tired of it. Oh, okay, I get it. Right now. And let's actually let's let's play the game that leads us to multi nationals. Be against this. And I came up and the AI came up with no they wouldn't be against this. Why. Okay. The number was 29 it's a boon for air quality and HBC business. There's also other good reasons. And we're seeing it play out in the media right now. Republicans have turned on big business, because they're woke. They're doing media every day. Republicans, not the rhinos, the real Republicans are against big business being ESG and woke. Well, there's a reason why. And the reason is that multi nationals sell things to everybody. They need Democrats to buy their stuff. They need Republicans to buy their stuff. They need libertarians to buy their stuff. They need socialists to buy their stuff. Okay, they sell stuff. If they come out and be like, Oh, one way or the other, they lose half their market because of boycotts. We're not going to buy stuff from you anymore because you support X. If you're in the C suite of a multinational, that's not good for business. That's not good for your share price. Okay, so we actually rated multi nationals as a guess. Interesting now we've, we've blown up like Adam Grant. We've blown up the issue and now we're starting to parse it out. Go wait a minute. Corporate lobby is not just one thing. Yes or no, they're split. I see questions on people's faces. No, keep going. Okay. Let's get to the place where we're playing the game. Okay. Someone pick one. Pick one what a yes or a no or a constituent roll. Pick a roll. Yeah, pick a roll. Pick a roll any roll. One that's not already stated or numbered. How about logistics. Perfect. Sorry, logistics. Okay, now who are, who are logistics. You move people or packages. Okay, it's very. By the way, look around you right now. And everything that you see around you right now was brought to you by logistics. Everything. So it includes all shippers and transport companies. Right. Now this is very nuts and bolts. This is a very nuts and bolts business. We take a we bring it to be my first job at mobile oil was in logistics. So there we are. All right, so now what would logistics think about large employers keeping better. Well, maybe that's it. Maybe it's better. No, because of cost improve. Should they improve it. No, because of cost. But would they also get more business for having to like, ship around. Yeah, filters and, and ducks and What number is no because of cost. Is there a blanket just costs. Yeah, there is a filter cost, there's a filter cost. So I don't, I don't see a cost no. Well standards would be set on realistically high would be one maybe that's not what we're saying it's kind of a proxy or 22 or 32 landlords have to raise. 22 22 is very specific 32 is also very specific. It's about landlords online. It feels like there's a cost one 32 is the closest to me that it's going to add more cost to everything but I would think there would be a hey this is just going to add more cost to everything. But does. So from the logistics perspective what costs. Do they care about. Like, it's a labor basically upfront cost. Everywhere they have, everywhere they have people in the system, they would have to pay more money to keep the people safer. And pay more money one time to install better air systems and then a little bit incremental for power for those. Yeah, yeah, so that's the common fallacy, you know I don't want to spend the money upfront to reduce my long term costs, I just don't want to spend the money upfront, but you just said it's a fallacy, which I did not know. It's like the one that I, I, the one that Joanne and I talked about is prenatal care. It's like, you know, the people who don't want to change it's like I don't want to pay for prenatal care it costs money. It costs a lot of costs of sick kids and, and mothers health and blah blah blah costs a lot, not to have to work or emergency rooms emergency rooms are another one right it's like, I don't want to pay for people to get, you know, taking care of an emergency room, you know, and then you end up with a long term cost. This morning I was reading something that a surgeon had left hospital and when he quit the CEO of the hospital called means why are you leaving why are we losing so many young good surgeons. It costs them a quarter million dollars to replace surgeon. He said our physician assisted our physician assistants, PA like nurse nurse practitioner quit a year ago, you have replaced her we can't do our work without her in our clinic. So you're losing me you're losing some other people because you're not hiring one. And, and it's so couple years later, he looked into it and they still haven't replaced it and they still kept losing surgeons. So, in the hospital's mindset it wasn't worth spending the money for a nurse practitioner they preferred spending quarter million dollars replacing surgeons. So there's two different ways of framing the fallacy as you're describing it in my in my mind one of them is, hey, the problem is you're not looking at the long term costs and outcomes of the decision. The other way is oh but there actually is an upfront cost you're not, you're not getting rid of the upfront cost of a redesign reinstall or whatever. You're just saying that not doing that is stupid because long term. But those are two different ways of looking at it like, like when I said, you're going to have an upfront cost and there's some costs for for energy over time. Pete your response was yes but that's a fallacy and I was like, Oh, so they're so I don't need to spend all that money up front there's some other tricky way to to circumvent that, but that's not at all what you were saying. So, so in business you can always trade upfront costs for a long term financing right. So, well there's a more more pointedly. Businesses tend to make decisions based on quarters and management bonuses rather than, you know, long term financial concerns. So and it didn't mean to get us off into this tangent for this long but I'm just very curious about how all these things play out in the game. So, well you're playing the game right now that's just it you're trying to make sense, right we're sense making. Here's how I go ahead. Is, I was just thinking about the cost one for logistics the, the filter costs as prohibitive is that line. Like, is the HEP a two seven filter is that what's like going to be what would be put in buildings to keep, or like everywhere like buildings and cars or transport chasing trucks. I guess if it's actually if it's going to be like a Walmart, they're going to have something on the roof. So it's not, it's not going to be like a little box like you'd have inside his classroom, they'll have an industrial solution. That's one of the reasons against about no multi nationals going back to that one for multinational to sign a contract with a major builder HVAC operator to put filters in on the roof. It's not that big of a deal. They'll sign the contract those they'll finance it over 30 years or 20 years or sell or whatever. They'll get out of profits they'll capitalize it they'll write it off and appreciation I mean, it's just like a cost of doing business there's no big deal. It's a big deal if you're running a little store and you got one little store, and you got to spend 100 grand. Well that's, that's a different subject. So, so I think to, to grab that answer john I think what you're saying is 12 it's probably not a big deal for logistics. Joel that the reason I said cost right away for logistics is because I feel like logistics runs on pretty small margins and it's very competitive. So you're talking about people who run volume and yeah huge volume and low margins. So anytime you say something where you need to improve work is basically you need to improve worker safety. So all of all of that anytime you say that to a logistics persons that's going to they're going to say no it's cost too much, even if it's just a very incremental cost. You know they're going to think okay so that means I need to upgrade filters in railroad engines I need to upgrade filters and in trucker cows I need to upgrade you know blah blah blah it's just like way too much stuff going on. So if you added it up it might not be a huge amount, but it's just a knee jerk reaction anytime I'm in, you're asking me to improve worker, you know worker health safety issues. It's like blanket no don't do that kind of stuff, it's just going to keep you from getting your goods and services like you're supposed to be getting. So, let me tell you how I have the AI and I scored it, and you may agree or disagree I'm not saying it's right. I'm saying this is how we scored it. We scored it as being a yes. And the reason is is that every logistics company, people or packages have a big place where lots of people are either going somewhere, or they're picking and packing. And there's a lot of people in those situations that can pass the virus to each other, or whatever. If that's the case, and COVID hit our GDP by, I don't know what was it 10% not quite a depression that almost any business is probably going to be very happy to spend a couple of bucks to not have their business shut down, the next time the virus that comes by, or have or be seriously disrupted. So it's like buying a little bit of insurance. So in other words, putting an air handler on your roof is insurance against not having your operation running I give you an example one of my clients runs a business logistics business with about 70 people standing at a table, doing assembly work. They shut down three times for four days through the pandemic, because someone tested for COVID. Just shut down. Well, that's not good for business. If we can have an air handler, and people would test less positive less often. And just having a few extra days of business, or having some people that don't have COVID, coming to work to keep the business running is a really good thing. So we scored it as a yes. Now again, I'm not saying that that's right, we would have to verify that through polling. We're not there yet. We're not there yet. We can get there. Not that hard to ask them. The one thing that this process is highlighting is how many intricate questions it raises at each level, and how much each of these could use evidence of some sort or arguments of some sort or even proximity to people in those businesses. Like survey says, you know, this is like family feud in some sense, where, where you, if you could have like a serious version of family feud for this, then you could actually inquire with logistics managers, and you can imagine the opinion between logistics workers and logistics managers, and the big corporates who actually are sending money or not sending money to the logistics infrastructure. So those would have a very different opinion about, you know, what the outcome of this question would be. And the only people who sort of matter in this question the ones who have to have their fingers on the budget on a last. But now we're talking about excellent point. So, I may interrupt real quick john. So, we just got into a, you know, an interesting discussion and Jerry kind of said oh, we're kind of like, getting away from scoring and john said no guys you're doing sense making in our longer conversation with john joint and I, and john, we would, we would often get into like, like it was john brought up the example of merv 13 and and HEPA. I knew from a lot of stuff that cozy rosin ballboxes which use merv 13 filters are cheaper and cheaper and sometimes better than HEPA filters, but I didn't know that to express it it wasn't until john asked the question that we got into it. So we had that, that whole discussion about merv 13 and HEPA and, you know, are they really good enough. And it turns out that they are. I had never actually read any of those studies because I just, I just knew, you know that they were. I really appreciate the a lot of the game is actually learning new stuff Jerry kind of just like what you said you know there's, there's a bunch of rich dialogue that that that happens. The, the, the one of the best outputs of playing the game as we've been playing it is that rich dialogue. And, you know, so you. So instead of saying well we have to like make sure we score everything it's like actually we're doing a lot of sense making as we're just going to I really appreciate. What two things. One is think of this as a democratic republic but instead of having 50 states we have 128 roles. And this is kind of because I'm saying is would 51% of people in robotics, or sorry logistics before against it. Now, let me, let me give one more example of that. Normally you would think that the United Auto Workers and Ford Motor Company are enemies. GM's better. You would think the United Auto Workers and GM or enemies. Well they're not when they needed a government bailout. Whoa, wait a minute. Okay, I, oh, I see now. Okay. They're not there's no just enemies there's like it's all in it's all interconnected. We're seeing this now in automakers and another way. We're going to start building electric cars. Well they only they cost like 20% of what it cost to make a regular car. Jerry. Yeah, and in the chat I was putting in a little bit about long termism and short termism. And maybe a question for you john which is America seems to me to be the short termiest country on the planet just about. I'm a little trouble about this and conservatives will do anything to disinvest on prenatal care, preventative care, like you name it like infrastructure repair any of those kinds of things just go out the window, because somehow, you know, filthy looker and markets and capitalism are supposed to solve for these things, but they sort of don't. For me, one of the things that this process might ought to bubble out and say is hey one of the urgent things here if we're going to have better policies around air filtering is a shift somehow, and maybe a focused or targeted shift on the people who on the people in these roles who have these can these positions of authority to make decisions from short termism to long termism maybe even just around this set of issues. And maybe, and I don't know and I don't even know that it's the job of, or the goal of policy cues to suggest things like that, because that to me is a policy strategy result. I think it's already in here. So, yes, prepare now to slow the next pandemic is long termism. And herd immunity herd immunity is a shorter timeframe, government regulation is a very short timeframe. So, there is long and short term in here. The, what I wanted to point out was, I just lost it. Oh, right. Okay, so the AC side of the table is new businesses and technology. T C side of the table is established supply chains and jobs. And your Elon Musk, you're building electric cars, you made everybody else wanted need to build electric cars. Well, he's not aligned with multi nationals who got a good thing going right now and don't need things to change. We try to frame everything positively and policies. We try not to look at the shadow sides, at least on the game board. We try to keep very positive. So here's the point. Change for change sake is bad. Change for logic is good. But just changing stuff to change stuff is bad. And now, well here, I get up every day and I got to like live my life. So I got to go, you know, as my gas stations right there. And if he shut down. I'd be inconvenienced. If he stopped if gasoline got stopped being sold. I'd be inconvenienced right now because there aren't enough electric cars in the country. Grocery stores in Florida. Don't still don't I guess they just got toilet paper again. It's nice to go to the grocery store and get groceries. The supply chain needs to keep moving. So I don't really like change for change sake if it disrupts my life in a way that I can't. Okay, you get my point. So change and status quo are neutral terms. Now, if you're change biased, you think the status quo is a bad word. If your status quo bias, you think change is a bad word. See my point. We got a big disruption coming with insurance. Because it, it, it can be distributed it doesn't have to be in big corporations. It can be a little tiny companies with peer to peer and it can. I mean there's, you know, there's bad things happening with like hurricanes. But yes, it is Jerry. So this makes you think about change and status quo in a totally different way, and not ascribing malice to any of the roles on the table because guess what, we're the heroes of our own journey. And everybody on the boards pretty much needed I mean, there's an argument to be made that maybe realty. We need realtors. Maybe not any maybe in 10 or 20 years we won't but right now we do want to pick another role. I have a question before. Yeah, so you mentioned that like AI has also like helped pick an answer here. What, what kind of AI is that. Yeah, I've got right now there's two working modules. One is based on close ties, meaning what roles are most like each other, and what roles are most unlike each other. And I have it structured so that there's 256 sets of nine deadlocked, just like Congress. There's 128 that are five four and 128 that are four or five. And we look for non obvious yeses and noes. And the non obvious yeses and noes have to be scored by a human. And that's the condiment. Work and noise. So let the AI do the easy yeses easy knows what a human look for the obvious. But the computer do the obvious. And that's what the second set is based on. Sorry, so each role has like nine possible yeses and noes assigned for four kindred spirits and four enemies. Right. So on a like, not a like affinity and anti affinity. Like, like care give, I think one of them is a caregivers and gun owners. It's probably a high correlation to vote separately. What ones like, you know, believes in nurture and and self sacrifice the other one believes in tough love and and in the other one. So self love. Probably opposites. So we did, again, we did carefully 128. Yes, you know one way and 120 the other way. Then the second set of AI is based on loose ties. So this isn't what you're like with other roles. It's what all roles seem to be like on other aspects of life, like philosophy. Whether science or religion, that kind of stuff. So we have five scales of four choices each. Each role has a profile of the five person like a personality test, like the five things that best describe them. And then we find proxies for the 20 score the proxies and then let the computer fill in the rest. What we get is obvious yeses, obvious knows, and then noisy ones that have to be done by a human. Wait proxies foot. What do you mean by proxies foot? Well, I don't know if I have it on this. I don't have it on this one. So one aspect would be in your value system. Do you value reducing the suffering of others. So the proxy we used for that was caregivers was caregivers would be the proxy for for that and there were 19 others that were the proxy for like the poster child for that aspect of loose ties, science, religion, that kind of stuff. So it seems like so far the like loose ties and the tie ties, however you said that close ties and loose ties close ties. They seem consistent across each puzzle. Is that right like you use the same algorithm but then like where does the her puzzle logic come in do you like rank the question on which values are relevant or something like that. Well, all those two AIs do is check for somebody having their thumb on the scale. Right. I mean, if, if, if someone has a hyper partisan, and they're trying to tell me that private equity would be totally for higher indoor air quality standards, and they've spun a web to to confuse everybody that that's actually true, the AI would be like, no, I'm sorry, we check close ties we check loose ties we checked a board of balanced humans like my team were roughly were balanced on AC GT. So we have one of our editor, one of our team is an abundance person. Another one is a thrift person, another one's a commerce person, the governance person. So we have a calibrated to the, I think we talked about this, like five meetings ago there's a book called how to measure everything, anything, how to measure anything by Hubbard. So Hubbard talks about, if you calibrate humans and calibrate teams, they will make shockingly better forecasts. And this has been studied pretty good like in actuaries and how to make better decisions faster. So you're de biasing, which is what we're doing here is we're calibrating humans to be able to look at the world in a bow and a balanced way instead of a biased, a less biased way, a less biased way. Jerry. I'm three tiny questions that I put in the chat, PT versus FT means part time and full time employment or something different. Yeah, our time full time. Okay, that was not it. Oh, by the way, thank you for bringing that up. This would basically be. I'm going to make a couple of mistakes here, but this is basically the first quartile of earners, the second quartile of earners, the third quartile of earners and the fourth quartile of earners, roughly speaking. Okay, it's not perfect, but it's pretty close. What is 110 degree states is that very hot places. Yes. Damn, you're good. You're playing the game. And, and then do you have anywhere estimates of the size of each of these roles. Yeah, what percentage of the population because the weighting of the waiting. Yes, interesting by that. Right now that's part of the game. Will there be in two years or three years of Bible that has like, you know, all of that. Yes. Right now it's each player gets to do it when they're in discussions with other people trying to call the role let's do one gun owners. This is fascinating. Okay, everybody's first impression of gun owners is they're all Republicans. Well guess what, that's totally wrong. The research has done studies on this, and there are more independent and the Democrat gun owners and there are Republican gun owners. What that can't be. It is. It is. Okay, so if that's true, that would change nest that would change how gun owners might be called in these particular roles. Yes, there's still gun owners. They really really really really want to protect themselves and their property and their family and their friends. That's, that's good across all three of those otherwise you wouldn't have a gun. Why would you have a gun if you're not protecting your property or okay. But on the other hand that doesn't mean that they're all like raving Republican. Hard right individuals, because there's more Democrats and independence and there are Republican gun owners. The NRA is not really a proxy for gun owners. The proxy for the NRA. Okay, another one. This is great. Part timers. We check demographic studies. And it turns out that I'm going to be a little bit off here. Two thirds of all part timers are under 25 or over 50. And two thirds of them are women. Okay, does that help score the roles. Sure. Now we know that they're that's it's biased towards the feminine. And, and we've got their very their old, they're not middle and they're not in the middle. They're younger or older. Okay, well that's that's really interesting. There are more Republican governors than there are Democratic governors, just a fact of life. That's just, just a fact. So there's a piece of this that's very much like the political lifestyle like Gallup or Pew or even more towards political analysts I'm forgetting the name the guy who does a quarterly report that goes around and gets a lot of a lot of sites. A lot of views. But, but you think of the name of that one I'd like to know. Yeah, and then also like the Clifton values. Yeah, and a couple others, right. But there's a there's a bunch of places that try to create and then there's some marketing groups that try to create very cute names for emerging demographics and so forth. Yeah, so we used for independence. This is by Linda Killian, called swing vote. And you'll see high correlations she only has five. We came up with eight. There's a Republican study done that came up with seven kinds of Republicans. We sort of use that week paraphrased and changed a couple things. And I'm not, I don't remember how we, I think Democrats just fell into place because the Republican setup and the independent setup, kind of forced the Democrats into these particular roles. And then some are very obvious, of course, we got them oxen dubs. We have leadership. Oh, this is a fun one. So, there's Republicans that benefit from the spending when the Republicans are in charge. And there's Democrats who benefit from when Democrats are in charge. Maybe that's why they vote a certain direction. They're called party favors. What a great name for it, right? Oh my, my team's in we get party favors the next two years. Yay. Should we really listen to them. Well, we should listen to them. Yes, but we need to take it with a grain of salt with the other 126 roles, not just because you're the ones getting bribed. Right. Oh, you got swag. Great. We didn't. Okay. This is misunderstood. Well, all those big bad corporations are making so much money. Except about a third of the country's retirement funds are in those stocks. Wait a minute. No, it's not just the big bad corporations it's retirees who want to live off their savings before they die. 401ks and IRAs aren't bad. So the stock market's not all bad. But a lot of retirement funds got gutted by Raiders. That's true but if you look at, again, look at the data so if you look at returns for stocks, compared to the return for bonds over historic patterns. It's pretty close to two to one. So you'll make twice as much in the stock market than you will in the bond market. And sell at the right moment. Well, the point is if you buy and hold over time, it's two to one. It just depends on, yes, you can pick any date you want and go, oh, no, it's not. But if you take a meta study of stocks over bonds, it's two to one. So with the time you've got left, I don't feel like we've gone to the front door as a newbie player and had the experience that a newbie player has. Should we try that or is there a reason to do that? No, we can do another role. I mean, the point is when you run out of roles, you're done. No, no, no, what I'm saying is we're sitting here looking at the roles, but we're looking at the control panel behind the curtain, which is not fair. What a newbie gets is eight roles selected for them in a different interface, and they answer particular questions that you've set up for them, correct? Yeah, let's go there. Cool. But it should be like the same interface except that there's only eight roles displayed, right? We don't know. I'm not necessarily expecting the same interface at all. And if it were the same interface, I'd be pretty concerned actually because I have been puzzling through this interface now for multiple calls and multiple questions and have barely got a grasp of it. Okay, so this is what it looks like. Yeah, so here's, I mean, here's the game. So here's the setup. So you read that. Puzzle is on. This is kind of a different puzzle. Normally we have a policy to pass. We were experimenting with using it for a different, a different reason. And so we did who shares the blame for prices rising faster than wages, because you read the paper every day and everyone's going like this. Oh no, it's not our fault. I had someone someone last week was trying to make a case for Gen Z is causing inflation. It was like, Wow, I don't know how you got there. I mean, I just, I don't make any sense at all this by this puzzle took. Oh my God, this probably this was probably over 100 hours this puzzle. This was this was like a white whale. So we finally thought it was close enough. The problem with this too is you wait a month. It changes. So we just had to like publish it by post publish a version we can update it later. And you have a kind of a board of interns who help you create the yeses and the nos for each issue craft the issue description and craft the issue title and you do that kind of together except you're sort of the binomial dictator of the process. Well, right now. I'm the only master player. So, I mean, I have to be ultimately I have to be the editor because I'm the only person that, you know, was playing at that level. Because I've, for maybe no other reason that I've got probably my 10,000 hours in. Right. So, there is a certain art to composing these puzzles. There are very definitely is an art to it that may or may not be trainable is my perception right now. Well, it's a funny thing about. That's a funny thing. Who just made the numbers. I did. Oh no okay so I picked yes or no on this interface. And then when you hit submit. You get the answer key we've been looking at. So finalized answer key. So all the player does is click on each one as a yes or a no. And then when they submit, you're the get the because the family feud survey says shows up. Yeah, the because the because statement. Okay, so activists we had. Okay, so if there's a yellow bar. That's caution. Green for go gray for ghost yellow for caution. So we got activists wrong. I got activists wrong. The reason why is that activists are not responsible for prices rising faster than wages because 40% of full time workers are under the living wage. I'm confused about how you're reading this what did you get wrong. This is wrong because it's not a solid color. Solid means solid mean you got it right. We agreed. I'm sorry all the other colors are solid I don't know what you're saying here. Hello. There are grays and greens. Sorry, so the banner is orange or yellow, then we're wrong. Yeah yellow yellows for caution caution caution. Okay, I am not reading it that way whatsoever. I am completely missing the semiotics of your of your design here. So so so so so urban PT and national enters are both solid all the way. I've only just noticed that there's a difference to me I was looking at the gray and the green areas only the header color meant nothing to me until you just pointed it out. Yeah. So, I'm confused here I would have guess if I was doing this myself I would guess activists strongly feel 21. of activists as 22. So are you saying, I don't understand how you got, like... It's been one second, let me just clear, I just need to empty by cash. Yeah. Like, I don't understand how activists would get 22 and not 21. Like activists would be, it's greed. All right, let's just do it again. I'm just randomly picking numbers on that, doing anything. Okay. So... I just played the game and I decided that activists are to blame. Sanitation's not significant. I was reading this as what do activists think? Not as are activists to blame. But yeah, I thought the yes and no's was who shares a blame and the squares in the middle was what do these people think? So you're not saying what do activists think? You're not saying... Are activists to blame for raising prices fast, for prices rising fast to the energies? I was reading this as, what do gun owners think? What do care givers think? This is a different kind, this is a different kind of a puzzle. So it is, most puzzles are, should we do this? This puzzle was different. It was, I wanted to say, I wanted to try to help make some sense out of why we have inflation. Because it's way more complicated than what people think. I mean, oh my God. And in fact, it may be worse too, but that's another subject. So you just showed us an inverted policy key puzzle, where the roles are under question, not the questions. Okay, we didn't really understand that. And I probably shouldn't have showed you this puzzle because of that, because this is like a one-off. Maybe there's, we've done 50 puzzles and that are ready to publish. And there's two that are like this and they're all the others are like, should we do a? Gotcha. Yes or no? This case is, is our activists responsible for prices rising faster than inflation? Gotcha. And so here the orange banner, like on big tech 21, that orange- We disagreed. We disagreed. The editors and the player disagreed. And we had 21 that big tech is responsible for prices rising faster than inflation because they're maximizing profits. Okay. So I wish that greed was green and disagreed was red. Yeah, yes, I know. That's why this puzzle took a hundred hours because it was inverted and it blew my mind. And so again, I'm sorry that I just came up on our call today, but let me just point out a couple of interesting things that we found out. I call these aha moments. Planned obsolescence is great for increasing wages. Also sucks for increasing prices. It's great for wages. Why does it suck? Because workers need work. Workers have to have work. If you bought one car, you'd never have to buy another car again. How many auto workers would there be? Right. Why does it suck for prices? Pardon me. Why does it suck for prices? Well, because actually prices wasn't the exact correct word. Expense. So if I buy, I just noticed. I just noticed that- Prices go way up, but prices don't necessarily go down. This stopped working for me today. Now, if it's not the battery, I have to buy another one. Planned obsolescence, if it's not the battery. Well, that plant's gonna make more money off of me, and they can pay better wages and employ more people. Ooh, it's kind of a nasty way to look at economics, but it's how it works. I just agree with that. If you look at wages over the year, like wages aren't going up. The more they sell, the more people at the top are making more. That's not exactly true. So for 20 years up until COVID, and maybe 50, the only prices that rose faster than wages were healthcare, childcare, education costs, and housing in some markets. Wages were rising faster than all other expenses for about 20 years or more. COVID hit, it all changed. And we still don't know what's happening because COVID completely has rattled the system. And I don't know if anybody could say if they know what's going on. Now, were the wages rising fast enough to save the middle class? No, but they were rising very slowly, but they were rising faster than prices. Now, that would be another puzzle to solve. And we think we've solved that one in a couple of ways, but unfortunately during inflation, it's probably not the time to institute them. I'm impressed with the number of subtle and difficult issues each of these puzzles raise. And you don't get them until you actually can build a puzzle. Because building the puzzle is the sense-making. You have to do the work. I'm actually thinking like, it would be really cool if there was a, like this seems to be really good at building up the divergence factor of like, these are all the different ways to think about from all these different perspectives. I think, I don't know what would help, but just I'm thinking like how to take that information and now like act on it and like build something like constructive, I guess, out of results. This is the constructive part. This is the national idea leaderboard. These are all the puzzles that we've scored and published and their scores. So the highest rated score of 86% non-partisan rating is to dump the penny. Oh my God, what a waste. We're wasting $6 billion worth of zinc a year. We're taking it out of the ground and putting it in penny jars. So this is saying like 86% of the roles agree on an answer. That's our forecast, it would be 86%. Okay. John, I'm curious, what was the reason to not stop making the penny? Well, some people do infect hoard pennies. Oh, I have tons, big water charts. What's the reason to stop making them? Well, it costs way more to process, circulate a penny than the pennies worth. It costs two cents to make a penny. Well, that's dumb. Right. And you know, we're having shortage of goods and stuff and everybody's getting more expensive. Well, why are we wasting the inputs on things we don't need? Why don't we use the metal in something we do need and drive the cost down to a higher supply of pennies? I agree, I'm asking, what's the opposite side? What's the opposite argument? The biggest aha moment on the great penny meltdown was libertarians do not want digital currency. They don't want the government knowing what I bought. And although slippery slope is not a valid argument, it is an emotional argument. It's not a logical argument, it's an emotional argument. The slippery slope is we get rid of pennies, five years from now we'll get rid of nickels. So libertarians haven't figured out that the government has already slipped a chip inside of every penny? I mean, really, you'd think they were smarter. I missed that one. You'd think they were smarter. And it conducts, you know, it has enough copper and it's a conduct electricity too. See, done. We've reached the end of our lot of time. Do you want to do a little bit of debrief? How would we like to proceed? I'll just do a little wrap up. I did run the AI on this puzzle that we worked on today and we could do one of two things. I could just publish it and you could all read it and see what you think about what the AI and I came up with. Or we could save this and actually play the puzzle next week, whereas today was more of a get to know how the system works more than just actually ripping apart whether we should, large employers should increase their, the quality of their indoor air quality. That might have been the right way to say it too. Thoughts? Large employers increase the quality of their indoor air. Joanne and Pete, this is kind of your topic. I'd be happy to follow your lead. Yeah, what do you think? I'm good either way. I think personally, I think the sense doing community as a work should decide where to go from here with our understanding of policy keys, if that makes sense. If someone has another topic, we could take into that too. So if this group is meeting every week for policy keys reasons, then this was all we need to do for indoor air quality. But if this group is meeting to sense build, maybe not always using policy keys and maybe having a thing now with policy keys on, because we talked a lot about policy. So it depends on what the group wants to do. We talked a lot about policy keys here and not a lot of, like this, even just the 16 on each side, pros and cons and stuff like that or how. So it depends on where the group wants to go from here. I'd like to see us, I hate that it's a meta thing, but I'd like to see a sense make about what we learned by going through the policy keys exercise. I think we owe ourselves that. Go ahead, Stacy. I was gonna say, I think we miss an opportunity if we don't continue with policy keys because I think this first session was kind of split. Half of it was learning just about policy keys and half of it was actually doing it, which certain things did come out. And I think if we were to continue and just play the game in the course of conversation, more information would come out and we would be doing more sense-making. So I say, I think we should give it another week where we're not asking so many questions about how it's working, but just using it as a guide to actually have the conversation. So maybe we shortchanged the group when Peter and I met with John separately and we talked about the 16 reasons from con. And those were the ones that John ended up with at the end. Peter and I had more than 16 and they were a little bit different than what John wrote, but that was actually the conversation about indoor air quality and indoor filtration. So, and I realized now we probably shortchanged a group on that one. So we could. But we got to then talk more about policy keys because of- Right. So, but like I said, it depends on what the group wants. The group wants to understand air filtration, talking about the six reasons pro and con or what it is is a difference. So like I said, it depends on what the group wants or whether it's just about sense-building or whether it's about policy keys or what, where does the group wanna go? To be, I think sense-doing cares about sense-doing and tools for it, not about air quality sadly, but if we only have an hour and a half every week, I think we should spend it working on sense-doing in general rather than a particular topic. What about if we have another call? What about plan two calls separate times and ones for one and ones for the other? Like I personally would like to learn more about air filtration and maybe if we announce it, there'll be other people that show up. I don't know if John's willing to guide us in that using policy keys. I'd be interested in that at a different time. So in my corporate work and I ran, I used to run a hotel company and hotels and I had two meetings every week. What was the production meeting? How many chickens do we have to buy next week so that everybody gets a plate of food served to them when they need it? And the other meeting was about everything else. So it might be that this is two meetings. One is to work on and actually make sense of something the chickens and then another meeting to talk about everything else. I'm more interested in actually working on public policies than I am about a meta discussion about sense-making. I've already kind of been there and done that. I have a tool and I'm just using it. And Jerry, you said something earlier. Divergent and convergent. So if you don't have the question, you can still have a wonderful meeting trying to figure out what the question is. That's a divergent meeting. Well, once you have the question, then that's convergent. And now it's time to just make sense of it. And then you can adjust if you're making sense of something and realize that you need a questionary statement or there's four branches on the tree or whatever and you can adjust your path from that. But I'm mostly interested in just either having a meeting about what the question is or having a meeting about actually making sense of a topic. That's just my opinion. And when you say having a meeting and making sense of the topic, do you mean without policy keys or using policy keys? Oh, I don't necessarily need to use, I mean, you can use any tool. I think there's lots of other tools. I'm biased. This one is shockingly effective. Mere and dear to your heart. Well, look, I can knock a topic out in four hours. I can get our first draft of a puzzle in four hours and learn a lot about something I never learned anything about before. And then in edit, we learn more. But I don't know what other tools can get that kind of granularity in hours and not weeks, right? Joel, any strong feelings one way or the other? I'm definitely more interested in the sense doing than the topic, but that being said, I feel like anything we've mentioned so far, I feel like I could pick out valuable learnings on the meta aspect of it. So I will say I probably don't have time for two meetings a week, but I wouldn't feel too left out, especially at the record and I could watch stuff, but. Yeah, and can I just say it would be good if certain people only went to one meeting? Like I might probably only go to one meeting because sometimes it would slow down the other meeting to have somebody whose main interest it's not. I think that's a good thing. Yes, that's fair, that's fair. Yeah, if you're really interested in regenerative farming, we might get 60 people to show up for the regenerative farming puzzle, but next week they may not show up because the puzzle is on inflation. But also if 60 people show up for one that aren't interested in the meta sense doing, then the meta aspect would get thrown out the window in terms of like retrospective learnings or whatnot. Yeah, my hotel metaphor, we didn't invite the sales secretary to the production meeting to talk about how many chickens we're gonna buy because she doesn't need to be there. I wonder if next week we can talk about what the sense doing team is trying to accomplish because I'm not sure that we really have a good, we never really resolved I think what we're trying to accomplish. Sorry, I'll wait and tell everybody's complete on this and then I'd like to jump in. Go ahead, Joel. I'll just say I feel like we're probably, my impression is that we're a little split into two that like maybe the two meetings would make the most sense because I feel like there's some of us that care more about the sense doing and less and more about the topic. And maybe that would, like if we tried to align on like we want one goal to be the sense doing or one goal to be the topic, I don't know if we would be able to align on that, I guess that's what I'm wondering. But curious to try, I guess. Yeah, a couple of thoughts on this from me. This is my fourth, maybe fifth time seeing policy keys on screen, John and having you explain and guide and stuff like that. And I feel like I learned 80% of what I know about policy keys today. Which means a whole bunch of stuff that wasn't clear to me before just like, oh crap, really? That's how it works. And this is doing that, which is weird to me because this, like Chucky, like Nolan Bushnell who invented Pong basically said you have a quarter, you have a minute to get the next quarter, right? And this is not a game where you have a minute to get the next quarter. This is the, and what I like about it is that it complexifies and takes you down the rabbit hole and all that. But then there's a whole bunch of messy stuff about it. And I'd like not to have our sense doing calls become policy keys perfection calls like, how do we improve policy keys? Which is a separate goal that I think is noble and interesting. I'm not looking for how to make the game better. The game's good, the game, it's scrabble. I mean, we've already, it's set. What I am looking for is making the puzzles better. Yeah. And then I was unclear when we were actually playing the policy keys game because at some point in the middle, we were going deep into some of the roles and you were like, yep, we're playing the game. But then I was like, but we're not really playing the game. This interface is playing the game. And then at the end, we came outside and went in this room. And then I felt like we were sort of playing the game. So I was unclear about when that was happening. Well, one is digital and one's analog. It's the same game. It's just a different interface. Okay. And if anyone wanted to split up a different call and run policy keys on this topic with other people and see who shows up and all that totally thrilled, that would be awesome. I don't have time for two calls, as Joel said. I feel like we've sort of slid past the air filtering topic. Like we're looking at the cake as it was baked and pulled out of the oven. And I just missed all the, like first, you add some flour and some leavening and then there, you know, you're going to need some sugar and I recommend this kind of sugar. I missed that entirely. And I want to sense do around that. I do feel like we sense did here because the cake that we're seeing is one version of sense doing using a particular tool. And I love that. And I think that one of the things we may want to do is shift tools or have a call like the hoedown where we have multiple people with different tools taking a swing at the same exact issue and then comparing notes and saying, here's what I came up with and here's what I came up with and here's how these things tie together or a complimentary or whatever else. I would love to do more of that. Yeah, that'd be fun. And as Pete said, sort of a little more refinement of what do we mean by sense doing would do that. I think would give us a maybe more of a methodology or more of an approach or something like that. I don't know. But I'm, and this is very funny. I feel like I may be the last person on the call who is still extremely curious about this air-filting thing and MIR versus HEPA and CO2 meters. Like, oh, right, right, right. We need to learn a lot about CO2 metering and how does, and what role does that? Like, I have a hundred questions about air filtration, never mind even getting into the policy issues of whether you mandate it or suggest it or offer rebates or my brain is just full of that kind of stuff. And so I'd be excited to go back into it. And then modulo, this is not the topic I was hoping we would tackle first, which was masking around which I have the similar number of questions only I feel like I know more about masking than I know about air filtration. So in some sense, there's a lot more gaps to fill. So that's kind of where I am. I'd be very happy if we went back to air filtration for a bit and until we sort of had it more sense done, I guess, yeah. Well, just hunting for the yeses and the noes is high quality sense-making, right? Yes and no, except it doesn't systematically in some way go through the logic of why these things even exist as the yeses and the noes. So I feel like Peter and I were able to come up with the yeses and noes. Like I said, we had more and John boiled them down to these because Peter and I knew the topic. If you gave me a topic I knew nothing about or if I felt I didn't know enough about, I wouldn't be able to come up with the, so I think that's where Jerry is talking about. And I think you have to learn about it first before coming up with the yeses and noes. Well, you're trying to come up with the yeses and noes to do the research as well. Yep. Barbara Oakley wrote a book called Mind Shift. She says there's three different kinds of thinking. One is scanning and sorting. The other is divergent and the other is convergent. So I've used that in this model. So first you have to find the yeses and the noes. That's the scanning and sorting. And then you frame it around an instigating question, a divergent question, which may not be the right one and you may have to go back and change it. And then once you have the question and the yeses and the noes aligned pretty good, then you can converge on the game board. Well, the analogy might be this. You got most public policy conversations everywhere today are about three inches thick. They don't get, okay, this is taking one shockingly deep dive on one aspect of something. And the piece of this, sorry, John, you're not done. Keep going. No, I'm done. So the piece of this that maybe most excites me or thrills me or something like that is the long-term persistent conversations like this that build a shared memory that gets better over time the way Wikipedia seems to sort of get better over time that take advantage of static ever-improving memories like Wikipedia, but then take them up to a level where you can say, oh, because this, this, this, we think this is a policy result or a strategy to try or something like that. I'm very excited about that, which means that when it's like US is already top 10 nation for IAQ, I'm interested in how that connects back to the stats that tell us that and what that means relative to other countries and what good enough IAQ means. And that's another little tangential discussion. I'm interested in how all those things sort of fold together so that this is not an overwhelming question for some panel to do in 90 days every time, which is what happens, but rather a thing that gets better over time where we start dissolving the problems that are thorny in front of us because we patiently sort of chip away at the different aspects of it, if that makes sense. So in a weird way, I'm realizing from saying what I just said, John, that this being a deadline type game where you have X hours to set up the game and then I get to play it for a couple hours and then we're on to the next game or the next puzzle runs against that. For me, the questions you're posing in the puzzles are important questions we should sort of slowly be chewing at over time and maybe we shine the flashlight on one at a time, but each of them keeps getting better as we leave them in our wake. Right, the trick is to get to satisfying first and then go to optimizing next, right? But I'm going for satisfying. I'm going for let's get 80% of the problem done like that and then move on and then improve as time goes on with an edit. Another puzzle on the same subject, a slightly different question. Cool, Joel, you're going to say something? Well, I guess I was just going to comment that I feel like the tool policy keys is good as a brainstorming tool to like force yourself into different perspectives, but just, you know, earlier how I mentioned the output is like not like, I think we show the role, the agreeance that we kind of like came up with as the results, but maybe this is where Jerry is looking for more, but like that doesn't like convey how you got to those numbers and like what the actual reasons ultimately, like how justified each reason is or how they come together to like result this decision. So the way I see it is like, this is a great tool for like getting the information or like thinking about what information is there, but not for like consolidating and like, I don't know if synthesizing is the right word, but like calculating based off of that stuff. Well, I guess it helps you pull, like make a vote, but I'm also less interested in like the voting generally because I'm curious as the reasons. One of my interns keeps making the following point on that subject. Is this a believable narrative? And is it a more believable narrative than what you're reading anywhere else on that topic? Interesting. Because what article have you read this week that gave you 16 yeses and 16 noes on the subject that it was writing about? I would venture to say none. I know, but part of the problem with 16 on both sides always is the both sidesism issue that you kind of elegantly sidestep and that is very problematic. Well, because I'm taking, on this is cool, you got to a really important point. If we're debating each other and I'm trying to win and you're trying to win, the other sideism is a completely valid argument. That's not what we're doing here. We're trying to find out where we can agree. If you invert it and you're looking at only where we can agree, then the other sideism falls away because it's only the valid reasons, no spin, no vitriol, no name calling, no gossip rumors in Uwendo, fake news trash talk, exaggeration, retribution or sabotage. It's a good valid reason why it's not a good idea. It's mostly an urban problem. It is, it's mostly an urban problem. People don't want to acknowledge that, but it's true. Okay, so by doing it in the positive with a strong tone and taking away the spin and looking for agreement, it just builds a more believable narrative. I have a really quick question. Have you, are you familiar with the liberative polls and have you compared your process to the liberal polling? A little bit, yeah, I like Fischkin's work a lot. What I don't like about it is that he seems to take topics that are kind of like intractable and just trying to convince the other side how stupid they are or to change their mind because they're stupid. And I think, look, that has value because lots of times the other side is stupid. Actually, all the time the other side is stupid, but that goes for all sides. The question is, you have to make sense of this stuff to get to the real crux of the matter. And the other thing I don't like about deliberative democracy is that you ask right off the bat, hey, Jerry, what do you think about indoor air quality? And as soon as you say yes or no, you're anchored, sorry, you're not changing your mind. It's gonna take five times longer for me to change your mind because you've anchored. We're not anchoring here. We're just trying to say, look, should large employers do this or not? And I fear that I've just opened up another lovely kind of worms of discussion on the topic. That's, I love what you were just saying right now, but I've got a boogie. I'm happy to turn the con to anybody else that thought that people wanna stay on the call, otherwise we should fold and reconvene next week. And let's talk in between on the Sense Doing channel in Mattermost. I think that's going well too. And the three of you, thank you so much for all the work you put in to present this. You clearly had great conversations, thought about a lot of good stuff and it was very juicy for us. So I really appreciate that. Cool. And I'm probably gonna publish this puzzle, assuming it's completed soon, two weeks from today. Awesome. And policy keys. And put a post on the Mattermost channel for us when you do. Find, tweet about it so we can retweet it, tag us up, LinkedIn, wherever you wanna put it. Sweet. Cool. All right, this was fun. Thank you, everybody. Thank you very, very much. Bye-bye. Bye.