 Yeah, I so I've been looking at the process of polis. I don't know that I'm pronouncing it correctly That's right, and we'd love to learn what's happened since 2018 with them, you know, I should let Tristan drive But I was just asking about polis and kind of how it's gone in the last couple years Yeah, it is when bilingual we use it for diplomatic agenda setting We run for digital dialogues with the defect of US Embassy here at the AIC and We together made a set of 40 recommendations So that across for example making powering more unique in the world trade Relationships security cooperation people to people ties and that made The foreign service on both sides as well as the relevant ministries hold themselves accounts to talk with only the 10 most Powerful rough consensus that is agreed by the people and very interestingly the most divisive topics for example How I should make English a working language is usually just being new ones Kind of more eclectic and turned into a consensus item like how I should move toward becoming a bilingual as much As possible so so basically instead of now as soon as possible and now is divisive But as soon as possible is rough consensus. And so we see a lot of very interesting dynamic like that You can learn all about it in the digital dialogues page Oh, great. You have this is I was going to ask you because it's funny. I googled for your work But I didn't know where to dig in and so if you part of what my goals for this call would be just to have a sense of Key resources that best describe your work And So this is my uh, this is my office website And there are three click here for more just Below the what do we do? and so the first one Connects to the participation office and network, which is this internal horizontal network Where every ministry sends a team of people to work on open government and engagement And we have like monthly meetings and our own collaboration meetings That works on cross-ministerial issues And polis is just one of the tools that we use And the digital administration links to the civil iot, which is a common sensing platform using public ledger for Air quality water quality and so on with thousands now tens of thousands of citizen contributors It's one of the largest deployment the citizen science that actually gets into public administration decision making And finally participation governance There's a click here for more that links to the presidential hackathon where every year we choose five teams out of a hundred or so That's basically receive Incubation in the 20 team stage and those 20 teams were voted in through quadratic voting Which is a grand wide invention Well reinvention and then five teams receive a trophy from the president Which is a micro projector that when turned on shields the president's face promising the team that whatever they prototype In the past three months will become public policy within 12 months with all the personnel and budget and law required And so these three together are mechanisms for such innovation to lead Government change and that I think this one of the more viable ways For digital to make democracy more democratic now I'm just setting the kind of time frame. You have my undivided attention for two hours And so we don't have to be too rapid. We can just get to know each other better But if you have other commitments, like if you don't have two hours, don't worry. We can always reconnect I I definitely have two hours. I will make time I I have about an hour and five minutes from now. Um, so maybe you should drive the first hour Yeah Well, I mean I I hate to make you do kind of your the kindergarten version of your spiel and you talk very quickly And I know you're an engineer. So I sort of like, um, I don't want to bring you back to kind of the basics But I I'd love for us to orient a little bit of the groups or we're kind of speaking I'm fine with repeating the basics that that's how we know each other. Yeah, okay Um, and believe me, it will be worth it I'm very excited to bring your work to both You know, again the audience of people that we reach here in silicon valley because we kind of take You know with tech platforms as an audience and then also Governments that we are commonly communicating with and I want a better understanding of work because I'm mentioning it offhand But I'd like to understand the mechanics and especially how you got there because I know that you've built all these tools And so now you're operating in this world where it works. But what was the process by which you were a little bit there? Yeah Before we dive into that is make a little mini agenda of the kind of things that we're curious about Wanting to dig into the biggest questions that we have and I'm sorry for the noise Um, so let's go. No, it's fine. It's fine. I've got air pair pro Sorry air pots pro and it's fine canceling noise on both sides. So Not really, but um, I don't get bothered by noise that much at the point here Two points. First, I totally agree that maybe we just make a short agenda Just on the chattering on the things that you would like to explore in the next hour And the second is that in taiwan, uh, we call programming is translated as emendering program design Not software engineering. So I don't consider myself an engineer and by Translating that as a program design. We actually recruit more girls than boys because designers are mostly girls So now we have to incentivize boys to be interested in programming just a very interesting anecdote That's awesome. I love that. Um Yeah, great. Well, um Of course, do you want to brief? I mean, I know you've introduced yourself a little bit But maybe to talk about some of the big questions you have and how you're thinking about this I just view what I'm already doing Largest of the versions of all the things you think you don't care about. So The broad arc is I think a shared interest in how to use digital tools To facilitate democratic process and to actually do that well So, you know, the the the processes that you've put in place, you know, to some extent I want to learn a little bit about what you've learned as a result of implementing these things Um, and so that we can basically figure out ways to maybe if we can to try to influence similar things to happen here Um, my my interest is essentially very parallel to yours in a sense of trying to design Uh, the right kind of tools and infrastructure to support Uh, you know, large groups of people to make really good choices about critical critical things Um, so you know, you're a background depending on so that he has that context. Sorry. Sorry. So Audrey has that context, right? Well, I've I've I've been a contractor for the government. I When I say government, I'm talking United States military for the most part And so in effect, I was involved in a number of projects having to do a search engine design and Was the architect for a system that basically became fairly prominent among the different agencies To try to share information and to do Sense making kind of things Um And and so, you know, in a lot of ways the the interest that I had was essentially, you know World well-being right from a right from the beginning, although I think the United States itself had a somewhat smaller scope in terms of their own well-being But but just just so that you get a sense as to You know, I've been doing architecture and design in this space for a while as have you yourself and and basically there's a An interest in trying to to to learn kind of what the on the ground experience Of the of the sort of design concepts that have informed the tools and how that experience of using this in democracy has has has been informed over its actual applications So that that as I'm doing design in this same I think of design as as essentially maybe a similar way as you is to to think about both the relationship between humans and machines and and mostly between humans But but basically to be looking at how do we do You know governance, you know by the people for the people of the people in a way that technology enables us to get past some of the biases some of the kinds of social processes that in the west are Really disabling and that have have basically divided our country So so in effect we I I see the United States as misusing technological process to our own harm And that some of the things that you've been doing is essentially actually really good examples of how to use technology in ways that actually help So I'm trying to be sure that that I'm as informed my overall agenda in this conversation is to learn You know, both your thinking and the the sort of the experiential information that might not have Have have been as easy to convey in in in things online. So that that I make better choices trying to do similar things as you Okay, so that's design concepts and journey of pro-social democracy processes in digital governance context Do we do we have more? I would be very interested by Forrest just mentioned on how you address issues of cognitive bias and group dynamics So we know about like-minded groups gather together their views get more extreme if you let them Talk amongst themselves. Um, how do you uh related to that? I know you've sort of had this gamifying consensus or only put a different way anti polarization design patterns So ways to actually polarize how do you anti-polarize when that's already the case one of the problems We have here in the united states is that we've undergone an artificial amplification polarization process I'm really interested in how do you reverse that and I have a sense that you have a strong Experience about how to do that. Um, I know one of which is the game of That means more gamification by note that of consensus driven processes That's one of the other areas that Tristan so I I've I just literally spent a lot of time learning about the about some of the systems in this space But I don't know very much about them. So so how familiar are you with polis? A pol dot is uh system. Yes. Only through some of the big, you know, pbc articles and things like that that I've read Because I'm trying to basically also gauge like an agenda item would be for me would be to learn What other systems in addition to that What? Yeah, what you would do differently next time around Um, if you had to do it again, what would you do differently? Um, and then uh, Essentially, um, is there a way in which new systems or new work has done It has moved from statements to questions Yes, yes And then also, um, just to mentally make a list for me by the end of the call What are the things you wish the whole world and the facebook's 11 meter tech platforms of the world New not because like I think that facebook should implement everything you're doing but in the sense that um What's possible that people don't think is possible? Summarizing these lessons the ways that you would if you were to critique some of the existing platforms Because i'm kind of constructing a meta agenda of things that I would like to record with you in a podcast Environment later, so that's another thing i'm kind of tracking. So Okay, so uh like what's possible for fb to Uh, to further, I don't know civic integrity or something like that. Okay. Yeah, you were running facebook civic integrity team for 2.7 closer to three billion people now what What how might you think about the applicability of the things you've discovered and the things that you know, I would never do it This way I would think about it And the basic thinking and modeling that would inform that Um, would be really interesting to to learn. Okay fun fun anecdote I've actually interviewed with the facebook civic integrated team on this very topic and the whole transcript is available So, um, I'll make a short summary, but but it's it's there. Okay. Awesome. Awesome. Okay All right, so, um, yeah and anything else from forest well, like I said just the relationship between um questions and Statements like how you might have like if you had to do it again What would you do differently? Um in this in the sense like if given the knowledge that you have But what would be the things that you that you've observed would be oh, we want to account for this Given that we're starting more from a blank slate I mean, I'd hope that but it's not actually the case but but if we were starting from a blank slate What would be the key lessons that we would want to integrate to bring it up to the to the next level? Okay. Okay. Uh, well, that's that's literally what we're doing Today with this participation officer network Consensus meeting. Uh, and so with uh, we have a recursive process that continues to Improve ourselves. So I can share with you on that of course. So so a uh, like recursive public and And where to find them? Uh, okay. I'm sorry where to build them Okay, what ways can we be of support to you? That would be very helpful to know as well, obviously Okay, so um, all right So that's that's it. Uh, we'll we'll just quickly go through them and maybe uh, uh breath first way and then We can just get into like zoom into it. Uh, whenever you feel like if that's okay with you Yeah, okay, right. So, um, you see my, um, kind of Drawing, right? Yes. Uh, okay, this is great. Okay Zoom is still pretty reliable as things go. Okay, so, um, yeah, so Uh, the first thing is about the design concept, uh, of pro-social democracy processes in the digital governance context And the very simple way of saying this Is um, just look to look at my office And so this is literally my office. Uh, the social innovation lab And I firmly believe that a physical place like this with plenty of food That opens until 11 p.m. Is the root of everything and we tore down the walls here It used to be air force headquarter. Um, everybody can just walk in and have 40 minutes of my time On every Wednesday from 10 a.m. To the evenings. Uh, and just just Bringing with themselves the toys likes those self-driving tricycles and trying to fit those new technologies norms, uh, which no You know, nobody knows what what they are about. They're like aliens Into uh fitting the nearby market a flower market so that everybody can through open source and open innovation To change them into whatever way the society feels like so this sandbox Coupled with direct access to ministers. Um, as well as my radical transparency, uh, protocol, uh, which Shows that after I become the digital minister for the past three and a half years There's more than 5,000 people, uh in more than 1,000 meetings and each and every single one of them You can find a complete transcript which we Co-edit after 10 days, uh before publishing and in meetings such as with david bluf speaking for uber at the time We make sure that it's not only on the record. It's on 360 record So you can put on vr and relive The conversation and the whole point of doing this pro-socio democracy context is to ensure that anybody who make arguments That furthest their interest must make so within a global goals perspective because of radical transparency It doesn't pay to argue for their private lobbying interests that is only good for them But assign externalities to the society or environment rather because of radical transparency Everybody is kind of like a completing a puzzle to get it And the effective partnership are shaped this way because everybody including david bluf can only make pro-social arguments instead of anti-social arguments In this lobbying context. So this is our face-to-face context and that's kind of a section one. Yes Can you um explain so you're saying if i'm understanding you correctly the way that you're ensuring a pro-social Motivation and conversation is someone essentially they get to only index and saying i want to have a conversation with you about sdg number 17 Is that kind of what you're talking about or that's exactly right? So everything that we talk for uh with Is within this context So, um, basically, uh, for example david bluf, uh, tried to make an argument based on climate change mitigation And that will be 13, uh, or maybe he tried to make an argument saying that this will enable more rural places to have access To transportation that will be 11 And things like that So we ensure that everything we do is indexed in sdg manners and all those social innovation Organizations that come to visit me eventually. Uh, I encourage them to register so that at any given time you can see Any number of priorities within taiwan and for example, we will choose a nearby tauyuan City, uh, there's english version, uh that focus on these in their voluntary local Um, uh reports, uh, the bor, but also you can see that all the social innovation organizations, it could be co-ops It could be um associations, uh foundations and companies, uh, working on which, uh, sdgs, uh, there are and also their natural Counter parts in other parts of taiwan. So the whole point is to build an ecosystem based on those shared goals And that's not only on the 17 level but rather on the 169 level with concrete targets Um And how did you it's very interesting that you're I mean, it's like you got to pick an ontology So you're you're picking a value system and your the value system you're picking is stg Named table number global goals. Yes. Yeah Um, was there a reason you picked that as a basis and versus, you know, I mean, obviously it's a nice convenient, um shorthand and shared shorthand That's very fascinating Yeah, so the reason first is that it can enable me to describe my own work in just three simple numbers Right, so I'm working on 1718 reliable data 1717 cross sectoral international partnership and 176 open innovation And and this is kind of a universal language in which that I don't have to go back To the drawing board, uh, whenever I meet a policymaker I can just give them those three numbers and they can look them up itself Although we change the usual diagram of kind of muscle Pyramids into this band diagram to ensure that digital is what We bring digital to the society and bring together the society using the digital, which is something like this Rather than saying that we're somehow More spiritual or whatever, which is the usual schema. So we change the official u.s schema a little bit So that's the practical application now for the philosophy This is important in the context of taiwan because taiwan is not a u.m. Member Neither is we member of many other international organizations. So first of all that give us A unique perspective like if we're not players or maybe referees But the other point is that the u.m. Uh process ensured that all the geopolitical Contentional subjects such as indigenous Transitional justice are not within the sustainable goals That is to say the system of development goals is this politic free zone In which that every member of the u.m Agreed and as well as all the major groups that these are the things that has the most synergy instead of just You know canceling each other out in terms of force and so that ensures when we do international participations We can say um taiwan can help Um Which is our main hashtag by the way Um without worrying about uh the negative repercussions that it may have in other jurisdiction that may be on the table So this is literally my name card and that says taiwan can help That's awesome interesting So I hope that makes sense and I'll move to the digital part So, uh the pro-social democratic Context as you see is based on the idea of radical transparency meaning transparency at the root and behind radical transparency is this proud tradition of internet governance of rough consensus or humming and humming is seen in taiwan as a A very important word here, which is a common sense or common understanding Literally uh gong shi and this is um gong is literally shared and shi is understanding or sensing and and this Usually translates as consensus, but this is not consensus because in in english Consensus is something that you can sign your name on or and something that you prefer But shared understanding is only something you can live with Something that you agree not to work actively against And so it's a rough consensus. It's like very rough consensus and by aiming for a very rough consensus or a shared understanding This is a lower threshold for participation because we can always move a little bit forward Instead of just having to fight uh on the different positions Right right there. And so um and now this is a very good um time to bring polis Because in a digital pro-social setting The initial journey of our conversation Was in 2015 and as for us now. This was the uber x graph and so everybody at that time looks at this four clusters k-means clustering and two orthogonal access the x and y axis are Computed through principal component analysis to automatically highlight the most divisive Two dimensions of the preference space Dynamically like in real time So you see this reconfigured as you play the game Pressing agree or disagree on each other's statements and as you do you move to where People who feel like you but also the whole graph reconfigures to identify new clusters This is the same Netflix recommendation algorithm And so the two design concepts the first is no there's no reply button In all the tools that we use there is no reply button because with reply button comes trolls and name calling But without reply button There's less ways for trolls to grow and the second design concept is that we highlight diversity rather than headcount So if you mobilize five thousand people to vote exactly the same way Then maybe you get an additional zero here But you don't get a larger area because the area here measures the diversity of opinions not the headcount When you say here, sorry, were you gesturing at something and I missed it? I was looking away. Yes. Yes. I was I was gesturing at this polycomb Oh that one. Okay. So sorry The cluster The the size of the cluster The size of the cluster You don't reflect the people that the headcount so you said the The thing that denotes the importance or significance is the diversity of people agreeing number of people saying uh, or sorry, how would you express that exactly? Yes, so, um The area of each cluster depends on the diversity within that cluster instead of the headcount And at the end we hold ourself to account to talk about the intergroup consensus meaning That's only these statements on the left Is held as a accountable agenda for the Authorities competent authority to respond to through a multi stakeholder forum. We ignore These in terms of agenda So everybody knows which are the divisive statements where they only have to look at the news To see the divisive statements, but uh, we only focus our energy our attention on the Intergroup consensus meaning like this is a real conversation that's had in Kentucky USA and whether they identify as democrat or republican Everybody agrees that instead of science technology engineering and mass There needs to be odd In this picture that the arts are important part of stem And that has cross-party bipartisan actually really not really related to partisan support and if we hold ourself only As accounts for these issues the mayor that hold this multi stakeholder forum can very easily get legitimacy Without being swamped into this divisive statement So it's a way for the crowd to be aware of itself a reflective space That are automatically cross-social and digital governance is just a way to amplify this You can do this face to face But you can do this face to face with maybe only 100 people or 200 to be a good facilitator But this is a way that you can do easily 10 times or 100 times more people What how many people are participating in a process like this online? This is what's amazing about what you're doing is normally I have been skeptical about I'm sure of course we speak of this too, but sort of like wanting to keep things smaller scale and then barcades and recombining But you're In fact, you're doing this online without the face-to-face trust building or even of corresponding face-to-face Well, yeah, there is a live stream face-to-face multi stakeholder component at the end of this process essentially What we say is crowdsourced agenda setting We set the agenda for a face-to-face conversation through digital means, but the binding process is always a face-to-face Or well, you can do it through telecommunication, which I'll get to later, but asynchronous a high fidelity Conversation that's based on those crowdsourced part-time Like everybody have two minutes of kindness agenda. And so yes, so as for participation rate So our national participation platform, which is joined the gov.tw is both Proposals that is to say e petition is also regulatory pre-announcements is also oversight in terms of budget KPIs as well as the Delivery so that you can see very easily What are the most interesting projects like long-term health care sanitation social housing? And things like that and if you click into each and every one The system also makes recommendation like if you're interested in long-term health care Maybe you want to join these petitions and so on. So basically it combines What used to be like four different websites in the u.s. government like we the people regulations.gov and things like that and Into a single interface and this interface has more than 10 million Visitors in taiwan considering taiwan is just 23 million people that means half of the population And the most active age groups are around 15 years old and 60 to 65 years out Meaning that people who have more time on their hands and for the 15 years out No right to vote have more incentive to participate actively on the platform, but everybody In between also because in taiwan we have problem as human rights Also, I can't just keep a tab on what's happening without contributing too much of their time Kind of just following it and for those followers those visitors. We have 10 million out of 23 million. That's amazing I have one quick question of course. I'm jumping after but um, you said diversity obviously was the key sort of factor So how do what is your definition of diversity? Right. So here I say diversity of opinions, right? So diversity means any statement in a in this polis context any statement that elicits a different response pattern as compared to every other statement meaning that It's not just a synonym or just a party line But rather it carves out a different agree or disagree Pattern across the people so that people can see that they're not sharply divided along Existing lines, but can actually agree on the shared values despite their different positions and these value statements are the most valuable Wow So it's literally the diversity of the way that people respond to a statement Yeah, it's like a ideal diversity like biodiversity, right? It's the diversity in the idea escape So In the hyperspace of all possible patterns essentially the what what what didn't close is the most space That's that's exactly right. That's uh, exactly the case and also, um, this space is reflective. So I like Normal vote saying which is three bits every four years from everyone Those bits aren't on a kind of secret ballot was the design constraint But in this case everybody's output is Continuously being reflected in this space So you can see the graph change in real time and that actually is the most important part because it gets everybody Into this picture and this picture basically is a picture of democracy Meaning that people See that they are the same polity instead of being trapped in the filter bubbles because really people are a polity It's just the media and social media highlights more of their division instead of the consensus So one one question that I that I would love to just open just just to just explore because I I noticed that the the the the platform really hinges upon statements that people could or agree or disagree with What I've been playing with in some of my own work is how questions and statements interact with one another I wonder if there has been some way in which Questions as a kind of coordinating basis for conversations I have been integrated into this tool or other tools and and it's just it's an open question I'm not I'm not saying there's a right answer. I'm just I'm just I've that that's a place I've been wanting to look at and I'm wondering what you found there Yes, and we did and this is the issue mapping technology that we use as the next step from from polis So, um, so how how much time do you still have? I mean I'm good until um for another 40 minutes probably I I think it's just so interesting and important. I may cancel what I have next but I'll try to do my best Okay, all right. I'm really and so I'm I'm I'm so blown away. I mean I know you get lots of praise, but it's very very uh cool and important what you're doing I'm really yeah People and I wish the world to model what you're doing faster, which is why I'm so interested in getting word out about it. Um Okay, right. So, um, so here, uh, we see, uh, our own facilitation process Using issue mapping. Uh, and so Peter's uh, is our office the public digital innovation space And this is kind of the tool that we use, uh, that's, uh, to kind of Categorize our current projects our approach our vision And then this is the uh goal actual value explosive value artifacts um drawing, uh, actually There's bound to be English version. I'll get your English version, uh soon, but um, anyway, um, as you can see, um the point here, uh, is not to, um for statements to, um, kind of capture the preference space by its own But rather this is a way to categorize the statements into as I said how close, um to shared values there are if it's a truly shared value, then it's a goal If it is a value that you can assign kind of Preference to it is a actual value. Uh, if there's a value that you can communicate That's exposed value and there's not a value at all. Is it just a statement? Then it's a artifact And so this kind of concentric ring, um can be also expanded in two-dimensional space, uh, like this And this is actually what we usually do in our face-to-face meetings So by the time that we get to the face-to-face meetings the polis, um statements are no longer Just a cluster that way, but rather what we're essentially doing. Um, and I'm Sorry, I don't have an English example here, but I'll just let you, um, see this general shape Of things so so usually, um, I think I'll just show the legend So we categorize the individual statements Into the objective, uh, which is blue, uh, the reflective, um, which is yellow The ideational, uh, which is green, uh, and the decisional, uh, which is brown And the blockers, which is which are red. Um, and so the point here, uh, of this orid or focus conversation method Process is that people can easily see how those statements Relate to one another and you can move them in real time as new argument surface During this face-to-face meeting with online participation parts. And so what's actually, um, Done in practice is that whenever we have a New brainstorming or what we call collaboration meeting We have the participation offices first vote every month on the two things that they they would like to Do a conversation in a multi stakeholder way. For example, there was a case where People said the text filing experience is explosively hostile And then we invite everybody who complain about it join the gov. That tw like this is a designer that originally complained about it And then you see these individual statements join or on polis being laid on the table And we use a digital twin of that picture so that you can Then see online exactly what's being put on the table Layed out in the text filing case as a user journey as a mess And which covers the before and during and after text filing But then we can do the co-creation across each Column which is each step of the process So we redesigned the text filing system together and that's what won last time 98 of approval Which is unheard of in government technology and services Mostly because there's thousands of people who say I have contributed at least one post-it note To that's text filing system or in terms of a more complex stakeholder group You can see a much more complicated issue mapping idea So it's it's there's different ways like concentric circles trees And user journey how those post-it notes can be laid out But the um a simple thing that hosts them in a strunt is this underlying focus conversation method that lets people see How close to universal values and personal values and communication Communicable our values and artifacts there are in such a context And so people do like they're completing a puzzle together rather than just being Drone by those different statements into different dimensions So in a sense it's a dimensional reduction through this very deliberate way of building a issue context out of existing diversity I think you're muted first I'm gonna I'm gonna go over the recording of this and and try to make sure that I gather more of the nuance Is there a white paper or yes, there is something could could you I would love to learn more about that whole segment That you just talked about In terms of just like how the principles work together like what are the Like like you've just spoken some of the underlying ideas. I'm going to go over the recording So you don't have to repeat yourself, but this is of great interest to me and I would very much like to learn more about this Sure, so the the underlying technology as I said is issue mapping and focus conversation method and these two are very well established in the Literatures, but the particular way we're using it Is in the participation officer guidelines and there's also three conversations with tom atlee that I talked about the application and tom atlee himself is actually One of the pioneers in these methods And so if you read the tom atlee blocks, which is in info that v taiwan that tw that will Probably provide useful inputs in the kind of day-to-day Conversations and how we deploy them into conversations. I I actually know of tom atlee's work I I have a palette of his books rescued from from So he's he's been a colleague. I just The particular I didn't realize I haven't been tracking the intersection between the work that you're doing and the work that he's doing I knew that there was some overlap But this is very exciting to hear that that he has been directly informing this process So so at this point what I'm what I'm trying to basically learn is What ideas of his you have actually applied? If you could just send me links to those conversations that you had with him or I just did is is is in the chat room It's in the chat room Yeah, there is the second link if you scroll to the bottom, you will see tom atlee's photo Great I will grab that and use that. Um, that is very very encouraging. I'm very I'm very glad I'm glad to hear that because tom atlee's work has been At least in my mind not well Not not nearly well enough represented in the world. And so it's to hear that this is happening Is is incredibly encouraging. So just I just I'm really glad to hear Right, so we probably have to rejoin this is a conversation because of this time constraint thing Because I'm using my personal account, but it's okay. I think it's the same Zoom URL anyway, so I'll just briefly pause the recording and then we can just join it again It's the same link. Okay and I'll just copy the The the chat so that we don't lose the the chat room. Okay. So see you in a minute Yeah, so we'll cancel this one and we'll join again. Yeah, you can just Click cancel because it times out by its own I think in a in like 50 And we're back Okay Yeah, so I was I'm just I'm super glad to hear That that piece. So I'm I'm definitely you know, I'm I'm treating this as a kind of exploratory conversation Just so that I know what homework I should do so the next time we meet I am More respectful of your time and It's fine So so The next piece that I would love to hear a little bit more about So so if I were to take Like the entire Sort of action of the government. So so in other words, if I take the the whole process And I think about it in the sense of how much has been Basically Integrated as far as you know people and technology to Basically do binding choices on what actually happens in governance And and so the question is roughly What percentage of the overall governance process is now mediated through These technologies these platforms the sort of democratic engagement how much has has in the sense The direct democracy become part of the overall process of what we would call legislation or executive action right, so in taiwan We have a system where The president is directly elected and the president nominates the prime minister and which nominates the cabinet so we in the cabinet are Kind of removed twice From the parliamentary dynamics. So there's more independent ministers than ministers of any political party And especially in my level, which is horizontal ministers It's mostly independence and the horizontal ministers survive prime minister changes many of us survived cabinet regime presidential changes too And so the point here is that we're a comparatively politically neutral place to make such Conversations and that is essential. So I'm not saying that any other parliamentarian system can just copy our system directly It is possible if there is bipartisan support that in a purely presidential system like the us that this can be carried indefinitely Although I hope the us gets to that point as a point, but in any place that the point here is that 100 percent of the The regulations that we make from the administration, which is authorized by law and doesn't have to Be a congressional the approved as well as I think over half Of our proposed bills to the parliament goes through the process on join the gov that tw and the bills that didn't go to this process usually are ones that are technical amendments to existing laws that has no repercussions on international trade or just you know stakeholders that are Like we already know the stakeholders the stakeholders already are part of the lawmaking process But all regulations go through this process So in indeed it's just like regulations.gov In the us all the regulations need to go through this 60 days process too the only difference that we differ from the regulation.gov is that this has a group participation signaling component Whereas in regulation.gov is mainly people calling people copy and pasting the same comments people just You know sending noise and like astroturfing instead of signals And I can go on to the public commentary system in taiwan's case. We make sure that the government gets mostly signals and not noise and also Because we have this participation office in that one whenever there is a cross-ministerial issue like two pre-announced regulations that are at odds with each other Then we incorporate these different ministries into multistakeholder conversations that resolve those tensions and finally People can at any given time through e-petition if they get 5 000 signatures to put us up or to turn a different direction On those regulations as well as policies. And so that are the main differences compared to the us system But we use essentially the same legal underpinning that is the freedom of information as well as regulatory pre-announcement Just universalizing this mechanism That is exceptionally encouraging. Um, I'm definitely uh, there's there's certain inspiration there. I think that uh I'm definitely feeling from that. So in regards to the The sort of tools that you use so so you've mentioned this the sort of conversational Mapping I I don't remember that name given the issue the issue mapping tool the issue mapping tool, right? So there's so there's that's a that's a component. That's like a key technology that's used as part of the conversation We talked about polis briefly. What would be the other major tools that that are currently Sort of central focus in in terms of how the overall dynamic of the Of culture works. Okay. So the the other one that we use in addition to the Imi issue mapping As well as polis is called uh, slido And slido is a commercial tool. We didn't code it up ourselves and we use it like I maintain personally the the traditional chinese conversation translation of slido, uh, I mean the system And we use it for pretty much all the town hall style meetings that we use The thing it was town hall style meetings is that like imagine there's hundreds or even a thousand people in the audience The projector is somewhat far away. Everybody has a phone But the phone is not large enough to display issue mapping And unlike polis, you only have one hour or two You don't have three weeks for those ideas to reverberate So you need something that Kind of maps the issues and set priorities You need something that lets people signal, but you don't have the luxury of either space or screen For the proper polis. Imi process to run And so what do you do? Do you use slido and so slido? I had a conversation recently with the milan interaction Design conference where I literally had a convert a real-time conversation with a whole room by asking them to go on slido and just input whatever on their mind and Like each other's comments if any of them are likable By the audience and so with a huge amount of people in the audience You can kind of just feel the the collective will of the audience in real time And that is how then I basically change Every sentence that I say in midway in response to the trending Voices on the slido system. And so that enables a kind of synchronous Agenda setting that is like polis, but in a much more real-time strategy where polis is more like term-based strategy And so that is how I use slido and it's slido.com. You can easily see it online So I will spare you the the details, but we use slido very Slido slido.com and The other good thing about slido is that it allows for pseudonymous Commenting and also it has pretty good automatic filters to make sure that people ask questions only instead of just copy and pasting random exclamation marks so What slido is doing is I will project a Image of slido on the large projector where people can see the most trending as well as the newest Sentiments or questions and I just highlight them one at a time and then I just go through them and Respond to them in real time. So it's a way to have a conversation with the room if it's a synchronous setting The other tools which is say it and you already know that I already show it to you is Our archive keeping tool In which case you can see all the context of things and there's of course also 360 live streaming telepresence tele robotics hologram gods projection and things like that, but these are more trivial the the real collective intelligence tools are mostly issue mapping slido and polis and also the joint platform itself which has this Columns of pro and con arguments and again, you cannot reply to one another So, yeah, the unifying concept of those four tools join slido. Imi and polis is that there is no reply button So one of the things that that I hear very specifically in the no reply piece is Is first of all to kind of act as a counterance against trolls so to to prevent people from essentially just Using it as essentially an acceleration of motion for You know the most charismatic or the essentially the loudest or the most Yes, aggressive voices to sort of take over the town hall meeting. Yes, but there's it It feels like somehow there's also And I haven't looked at slido yet. So I don't know but Is there something that essentially means that that that what a person is writing is not necessarily becoming public to the whole group? Or is it like what what prevents a person from essentially using it as a kind of private publishing media? To sort of try to take over With with conversation take over the conversation using kind of rhetoric devices or ad hominem or the kind of things that normally to rail Uh groups of people from not being like completely triggered and and excited into some sort of mob frenzy Yeah, I want to add on to that question before you answer it. Um, yes I had was um, how you accomplish what you do I don't know if you have a different kind of id passport type system or people have to log in ever meant id Like how do you prevent one of the big issues with the us-based tech platforms is that there's just Really just lots of bad actors starting, you know, there's as you know, there's dark markets Where you can buy vintage early virgin, you know, twitter accounts from the account creation dates of 2009 And you can just buy these things by the by the millions and then Um, you could just you can any the point being because we can't verify id That creates all these other problems and what i'm curious about is do you also have a lockdown trust id system? Or is like the kind of social design elements of no reply buttons, etc Enough to kind of just disincentivize Bad actors in other words, you could have millions of people registering millions fake accounts But they just won't get very far based on the social design system Which is fascinating because essentially you're saying without an id you can do it So that's kind of I wanted to add on to that. How do you just prevent the kind of bad dynamics hurting muscle conjugation outrage? Yeah, uh, both very good questions So, um, I'm assuming is actually trying to gain your systems as best they can Trying to manipulate your process How do you deal with that? Oh, yeah, well, um, it's uh, actually harder if you run this on facebook But we also have a playbook for for facebook because facebook it's it's not our own design, right? It's tilted toward a truce ways Um, and but what we we can um, we have a playbook that uh takes care of even facebook But uh, let's let's start from the the easy parts. So, uh, like, um, so Let me just show you a real slide of conversation because that's um always easier. Um, the slide of conversation that you see now, uh, for some value of now, uh, is Uh, the one that I had um in Osaka That is to say, uh in defcon And so it's less, um It's less polarized for obvious reasons because these are Ethereum hackers But you probably can see that there is still some diversity in the questions, uh, that's asked Uh, and some of them quite politically polarizing, uh, and some of them just, you know, um Pretty random, um, and so on and so the the point of the the slido Interface is that there is just so much visual bandwidth a limited visual bandwidth Um, so There are things that are kind of immediate, uh, is the kind of Motions that, um, for example, there's one that says turn off light, please Because the light before the projector, um, is a little bit, uh, strong And so people on the back cannot see the projector and for these we just, uh, because they will appear first on the latest question Column, which is the column at the sorry the row, uh in the bottom So what we do is essentially just turn off the light, uh by highlighting that particular statement, uh, which flows it To the top, uh, and I hope you can see how this works in real time I just click this up And then it goes to the top and everybody can see that this kind of takes over the agenda in an emergency And then I just, uh, turn off the light and once I turn off the light I can archive it and so people see that the light is turned off and you have a better Visibility into the statements and people who join the room later can always scan the QR code Uh, and join the conversation. So there's no easy early mover advantage to dominate conversation because the late commerce can always just post their latest question for other people to build on and, uh, People who ask follow-up questions, uh, only have one line to ask So to answer for this question if somebody use it as their blog It will actually be counterproductive for them because we only see the last line of the blog. There's also a, uh, Limit on characters. Uh, so it's like a tweet, uh, and only the latest one gets featured on the bottom So it doesn't pay to spam is what i'm saying. Uh, and finally, uh, the top, uh, voted in, uh, questions You can see i'm sorry You're saying oh, sorry. There's just a delay. I it doesn't pay to what doesn't pay to what it doesn't pay to spam To to spam the board or to spam it. Yeah, you got it Yeah, the spamming doesn't pay because spamming don't get you votes And if you don't get votes, you don't get this, uh placement on the top because it's Of the the the real estate dedicated to the things that are voted above the fold specifically in the very little placement given to Because how do you have people scanning the kind of latest questions to sort of it's on their phone. It's down their phone It's not using the projector Oh, I see. So this is a real one right now. Can I actually join join slido.com and On my phone now if you if you I don't know scan the QR code you can probably join it. Although, um, Let's see if if it's still active. I can easily reactivate that just a second um And and we also use slido for uh, like real-time voting and polling. So, um, this, uh, real meetings across different 32 ministries participation officers are also conducted through slido, which is important because in the same room if you have a senior officer and a Uh, junior officer reporting to the senior officer, the junior officer would not raise their hand and say something But using slido and because it's pseudonymous everybody is using their phone and the junior officer can Just input whatever pertinent information without worrying about your senior officer identifying them So a sense of pseudonymity within The cabinet within the executive branch is also designed by using slido as a Kind of ambient calm technology That brings a different flavor to the meetings. So now i'm extending the Date of the radical exchange conversation. So hopefully you can Just let me just Change the name of it to add more bits to it kind of literally So now you can join And the link would be slido.com Slash 1010101 Now because I just use the qr code and it worked for me. Well, yeah, the character always works Yeah, okay, and and so so yeah, and now you can start asking questions and and your questions will actually go to the recent column Uh, recent line and so you just say anything and you can be anonymous pseudonymous real name you name it Yeah, I just added a question Yeah, how can I make uh, Audrey president of taiwan? I regret to inform you that the taiwan constitution say one has to be 40 years out to be president And i'm not 40 years out yet. Uh, and so this is currently unconstitutional And by answering this I will just uh, archive it and you can see it being Archived so so that's how it works in practice And so people don't try to flood the system and just start posting lots and lots of questions Because I can make the list so much longer than people are able to read I'm just thinking about all the attack factors in which people try to Buck up the system. That's what I'm trying to figure out. Yeah, sure sure sure But uh, there's three factors working against that behavior. First everybody is in the same room So if somebody use a and there's only so much spamming you can do is your phone So if somebody types like crazy into their laptop, uh, the people sitting next to them will probably discover And so that's the the first thing uh, and the second thing is that if you make a list longer It actually sorts automatically by popular parts. And so, um, it doesn't really change the The top questions. That's the second uh thing and the third thing is that the moderator is at Anytime anyway in control. So, um, if I detect spamming going on I can just with a Just like literally in a second to switch It's from a live conversation into a moderated conversation or I can just Have a latest question turned off just by this and then you don't see the latest questions anymore or I can turn on this idea of Questions need to be questions. And so anything then that you reply that are not questions Will be moderated away and you have to actually share a genuine question and things like that So there's there's many different knobs depending on the spamming level And for example now I've turned it into Um, moderation only uh, and so your new questions don't go in anymore. Um, and so on And so that the point here is that, um The uh mechanism uh can be adjusted in real time in response to the audience behavior And that implicitly discourages spamming behavior This is I can like pause. Oh, sorry for us. I was just saying so I can as a moderator I can pause the submission of new questions. Yeah, it's already done. So so now if you enter something It first goes to the moderation queue Oh, I see got it and it will say awaiting moderation or awaiting review or something like that. So and finally for identity For different binding power. We have different Identity schemes for a slide though. You have to show up there to see the QR code So that's identity right there For join, you have to have a taiwan sms number And the telecom probably did a kyc In a very beginning If this has legal binding power like a referendum, then you need to use a properly state issued pki card And so it's just a spectrum of solutions Pki card. Is that is that the taiwan? Yeah, it's no pki is just probably key Infrastructure and we call it the eid or the electronic id is just this ic card Which also can be virtualized now into your phone using phyto technology And I think one in four people in taiwan have such a card Uh-huh. Was it one in taiwan have what kind of a pki? Yeah, you can just search for for eid. I think the the proper name is eid eid Just one really quick quick question in regards to the To the presentation device here when it goes to a queue for moderation Are those moderators on the site where the event's happening or are they off-site and part of the company that's running sido? So, uh, of course, it's on site. Uh, this company doesn't provide moderation as a service like in my public conversations It's usually me moderating using another device But uh, if the volume is too large sometime islands enlist a couple of colleagues who does the moderation on a pre-agreed Way, and that's how we moderate for example a very popular Life show with a stand-up comedian And the conversation which is also slightly driven Is quite something to behold because people really want to troll the system But our moderators Really, um, just keep the pace and so people eventually surface very insightful questions When you when when you're using the system, um, what percentage of the population is is typically asking questions So if I had a group of of 500 people coming to a town hall What sort of volume of questions is that generating for you? In other words, is it that It's it's a very typical wiki pdf curve, right? Everybody votes up vote Uh, like 90 percent of people purchase being the voting part. Maybe 10 percent Do actual ask questions and I answer maybe only one percent of the total amount of people So you'll have uh 500 people. Maybe I end up answering only question from five people The process of answering is is done in person. So you're just basically responding at the microphone real time Do the question Do the questions go somewhere and you respond to them at some future point? Or is there is there some way in which the questions get archived and treated as it's it's always archive here anybody who has the QR code can go back and see the questions and in uh, the social innovation tours, uh, Which is one particular application of the slido we ensure that all the competent authorities get to the slido questions After the fact and publish it on the si social innovation that taiwan platform So in a sense, um, the unanswered slido questions nevertheless become part of the meeting and you can see, um, a real, um A way of how that works in practice. For example, I would go every other tuesday And some fridays to the rural indigenous and offshore islands and people there Usually people in their sixties as you can see here Just control the agenda reporting whatever they want to say From their local vicinity. Sometimes I go there and stay for a day or two in an ethnographic Hanging out with the people but when they do so there's a zoom link Uh through cultural translators. This is the indigenous teacher doing cultural translation And also linking to the five municipalities including paipae where all the 12 Central government ministries, uh, usually section chiefs, uh, are in the same room literally So they all look at the same slido board and respond as they can on slido using hyperlinks and also I cue them uh from this facilitators role I'm the only one that travels they are online But I can cue any public servants in any municipal connecting places So that we feel like we're in the same room and the people there see the public service Responding to the questions here and now thereby increasing trust between them And the public servants don't have to worry about upsetting the local people because you cannot Hurt people over a projection. I'm the only one at risk And so the point here is if there are slido questions that are left unanswered because of time Because if it's cross-ministerial business that cannot be resolved right on the spot, then you can go to si.taiwan and see Where the slido question went and we send an email to everybody Who proposed such questions? So everybody can see the kind of proposals and the social innovation tours and see a database of what slido questions are currently awaiting Ministry of input and usually within 14 days We will yeah 14 days 10 working days We will make sure that these questions gets answered on the si platform by the competent authority So there's also a follow-up component in addition to the agenda setting in a phase two phase Consultation this this is this is wonderful that piece of it's sounding a lot like some of the work that i'm currently Doing with this thing called a femoral group process, which is another Another whole thing. I don't necessarily need to take up your time with that per se, but One of the key pieces that would be the next Sort of layer is that after the questions are answered you you're providing a way for people to find out that those questions are answered So they're getting a notification via email. Yeah, they're seeing this is they're seeing the response Is there some dynamic that that allows for You know like a kind of and this is I know you don't want to do follow-up in the sense of providing a way for people to You know again, you know once once again Up to the next meeting I mean or they can just find me on wednesdays that the whole point of this continuous integration Is that people can cite a previous unresolved issue and bring it either to the next town hall in their vicinity Or travel to Taipei or arrange a video conference with me and talk for 40 minutes at the time on the record That that sounds great. So but but the questions themselves so so you actually just answered the question Which was essentially the idea that the answers that were provided to the questions of the previous meeting Are in public records somewhere and others so the people in future meetings can cite the The responses from the previous meetings with their new questions in the future meetings Yes, and and the seo is pretty good And the reason why the seo is pretty good is that each of those questions and answers Literally have their own page like not not really Not only that This own page, but it's a full-take searchable. So if you want to know anything related to tom atley, for example, I'll just search for atley And you can see tom atley's input into our process and actually all the Places where I mentioned atley as well as colin and micky kashdan That was brought with tom atley to conversations And as you can see each individual line has its own link if you click into it, that's a page of its own That's uh, that's that's amazing. I'm I'm definitely uh, very very interested So this is this is this is I'm the basically tristan what what what they've implemented is is something like what my first two layers Of of the process that that I've been basically in development of and and we're looking at a A manifestation of that actually in progress one very specific question so A lot of the responses if they're coming from meetings or in conversation They're in video and such like that But we have a full-text search that's going on here. So somewhere There's a An audio to text transcription in the process. How is that managed? Well, it's a combination of assistive intelligence and human intelligence In places where the microphone is really good. We just use a standard off the shelf Speech to text technologies and then with co-editing the co-editing is key because everybody receive a collaborative editor that is server security hardened And so people know that these draft meeting notes do not goes to public. So you can see for example After a conversation with the christian science monitor, this is already published. I'm allowed to show it to you You can see that they Basically gets a link that Then for example, oh, there's a recent interview with rana furuha For financial times and that's I think a a better example So then rana after the conversation with me receives a link to here, but this is A not a access control list. This is a share token So I can see who have access it and rana and I are the only editors And so you'll see this conversation In a read-only mode that says this will be released on February 23rd So that's to say two weeks after the conversation and on a public into the commons Meaning that is relinquishing copyright and we thank them for the contribution And then rana can then just click edit and just start editing the markdown Of the transcript to fix any issues detected by her from our speech-to-text system And so that that's it. And once she finishes editing Then we just publish on this prescribed date and in many different Conversations where there is no such a good microphone. We also have professional stenographers Court reporters that transcribe this but of course there are also online services like casting words, which we also use And so the upshot is that people go through each other's words much more carefully in a asynchronous mode after the meeting again So if you're busy talking at a meeting, you can nevertheless experience the whole conversation Afterwards using reading speed and that also increases rough consensus I've had many times where the ministry will delegate after reading through the notes Telling me minister. We don't have to hold another meeting anymore. I actually got the other side's point I mean they didn't able to go back and read it again. They get the point that they didn't get That's right. That's right. Yes This is uh, yeah, this is this is this is actually exactly right So so what you've just basically demonstrated is is is something that I've been hypothesizing And it's it's it's very affirming to see this work I I think that maybe uh one one just sort of kind of logistics question is so You know when you're holding a meeting with specific people that you that you know who to ask to be part of that conversation Is are there circumstances where you have? You know people that are responding in a meeting. You don't happen to know who they are or there's a there's a kind of anonymity factor In the audio part Is there a crew somewhere another that does you know some sort of Consensus, this is the best transcript we can create. Is there is there a sort of background process for that that's Mm-hmm. Yeah There's a great question too. So um, we've tried different ways, but the long story short what we are now doing is essentially Each party contributes their own recording Which is easy actually if you have a zoom like system where Each side do a recording themselves. Anyway, so by hitting a zoom record You get audio streams even if the live streaming isn't working quite perfectly and on a meeting place We usually use a centralized microphone system and then we just record whatever that's going into the microphone So we also have a crew that reminds everybody to please use the microphone Otherwise, their voice doesn't get into the transcript. Um, and so um and in even larger meetings We have pushed to talk and and that's what we settled At the moment if there is no microphone system centralized going on We discovered that it's too easy to misrepresent each other's words So so this this whole constellation of of capacity. So you have a basically a collection of software tools I feel like I've got some of the arc of the overall flow of the process of how meetings are You know partially in person and then there's essentially this online component There's sort of discovery of common interests and how that infuses This this this dynamic is is is very good Is there some um like how does the government Support or how does the public support? Is there a tax base that's used to support the servers? That are hosting the software. What are the logistics that that provide context for the meeting spaces or You know to pay for transcriptionists to do the thing or to pay for those How it is is this a How does that happen in the in the overall flow of of things? Is this a Essentially an extra governmental process that's being integrated or is the government itself now doing this as as part of its own function right, so um the b-taiwan Which is a social sector process Specializes in finding zero cost ways to accomplish all this By relying on crowdsourcing and artificial intelligence or as I prefer assistive intelligence the government being slightly more stable Allocates dedicated funding for the process within the government And so if you go to the po website that I pasted you you can see not only the participation offices But also the enabling regulations And so the enabling regulations basically say that each Ministry that is competent authority Pays for the logistic costs and the office of science and technology and the national development council Pays for the hosting of the system and the service of the system, which is not a large budget by the way Once you have this system ready. It's um as simple It's not more expensive than paying for the transportation and accommodation for people to travel to a Citizens assembly is actually cheaper and so But you can see the directions and principles I think It's called topics process of how those topics are selected and the competent authorities how they assign the issue within them and who are Responsible for which supporting the structure and so on it's all on the website Um, I've got like maybe if just a few more questions. I don't want to take up too much time here But this is this is turning out to be actually quite important So we have this sort of discovery of communities. We have a kind of record of the conversations that have happened I mean, I really love the transparency of this. This is by the way excellent How does it move from these conversations? Into legislation or into budgets or into like what what ensures that there is a kind of consistency Between the the the sort of agreements that have been created in the conversations You know multi stakeholder perspective and so on so forth There's a transition where there's a kind of place where The community itself says yes, we have a we have a common vision as to next steps Well, it's easy. I talk I take these and talk to the prime minister or the minister is in charge So in other words these these these conversations become there's there's a transition From the the conversational content into essentially the traditional governance structure Where proposals are made and budgets are essentially defined But that's a specialist group that's essentially composing the exact language of the of the policies in the budgets Exactly, and that's why my office is delegates from each ministry So in Taiwan there's 32 vertical ministries each was one vertical minister and I Say any volunteer from any ministry? One ministry each can join my office as kind of official delegates And this is the core group in addition to the peripheral Participation of this network And so I must say not all through 32 I have sent people I have 20 colleagues Meaning they're not everybody sent people the minister of defense never sent anyone So our binding power to the ministry of defense is weaker But the minister of foreign service for example on public diplomacy Sends people and the people they send are usually around section chief level But for for example the national communication Commission is a director general And so every delegate bring with them their own political cloud And by making sure that these processes are co-created by the career public service delegates We make perito improvements in the sense that all these rough consensus with their input is guaranteed to reduce their risk or to Reduce their churn to save them time or to improve their credit credibility improve their trust Not all in three in the same time, but we never trade one for the other two So they are always perito improvements along the risk the time saving and the credibility Access and so once they propose such a thing to their minister, they're still reporting to their minister not to me I'm just a facilitator They make a very compelling case. Uh, and then their minister will just take these Reverse mentorship And from the civil society And then bring it into the ministry or agenda which for petitions For example, we require the ministers to respond within two months in a point by point basis of which Consensus made into the regulation which consensus needs time to wait Usually because of budget cycles and which are against the laws of physics. So we cannot implement them To is when when they when a budget or a policy is brought up to the ministry level? So there's an authorship thing that's happened There's some reflection back to the sense making that was done by the By the polis and by the conversations in the documentation that's been created Um, is there a ratification? Um of that policy that effectively involves the direct vote of the entire constituency or is it a vote of the Only only in this case of presidential hackers