 OK, that's good. Wonderful. So maybe we should go around to a round of introductions. Deb, you want to start? Yes, hi. I'm Deb Roy. I'm on the faculty of MIT. I currently serve as executive director of the MIT Media Lab and also director of a newly formed center for constructive communication, which is going to be my main research home going forward. Very nice to meet you. That's awesome, yeah. Nice. Hi, I'm Wes Chow. I've been working with Deb in various capacities for a few years now, from, I think, close to four years. And so now I run an engineering group inside of Deb's group that looks at the research that the center is doing and helps to play it out to partners and collaborators. Excellent. Thank you. You can get your emissaries back down, and I'm now, I think since we talked last, I'm actually now the associate director of civic design at the center and I'm joining. So we've actually connected even more. That's excellent. So I'm Audrey Tang, talent station minister in charge of social innovation, open government, and youth engagement. I totally agree with this idea of a kind of norm-based struggle. And my current work mostly centers around what I call people-public-private partnerships, meaning that the social sector sets the norm, the public sector amplifies the norm, and then the private sector implements the norm. This is rather different from the more traditional, so-called public-private partnerships. I love that, I love that. Well, I wonder if we can start there, which I'm kind of introducing, starting to work there during C3, but we're up to. Happy to. Audrey, I have a few slides just to increase the center briefly and then show you one particular project we thought might be of interest. So that's OK. Please do. Please do. I mean, and show you a bit. So the new center we announced in January of this year is called Center for Constructive Communication. Can you see my screen there? Yes, very clearly with the three circles becoming a triangle. Great. And the challenge that the center is setting out to work on is one of social fragmentation. So we observe that in various realms, including politics at the workplace, on the airwaves, online, on the streets, and sometimes even at home, we sense an alarming social fracture. And our aim is to make a problem that's easy to recognize, possible to understand, and hopefully, in some ways, address. We got into this space, which is, of course, multi-dimensional and complex with work we've been doing over the years and analyzing social media. So I'll just show you briefly two examples. This is a snapshot of Twitter users during the 2016 presidential election. These are accounts that we believe are primarily humans, not bots, that were actively tweeting about the election, where the graph is formed by finding mutual follows, which tend to be people who know one another, more likely to know one another. And the color coding shows followers of candidate Trump in red, candidate Clinton in blue, and candidate Sanders in green. It's a picture of a fragmented network, very weak on activity, relatively speaking, between. By the way, can you see my mouse? Yes. OK. Well, your cursor, not your mouse, but yes. You can see my pointer. And I now realize I must be very precise with my words, too. And then we also became curious about the location of journalists. So we built a database of the journalists who had written at least one story about the election in a national-scale publication in the US. And shown in blue are verified in yellow, unverified location of the accounts of the journalists. Again, all trapped in one side of this fragmented network. So this image became for us kind of iconic, though, the problem of fragmentation. One more example, just to give you a sense of what we're doing that led to the center. This is another data visualization. In this case, in orange, the retweet cascade shown over that plays out over time of how a piece of news, which fact-checkers later found to be false, propagates through the Twitterverse. And in blue, a story that's fact-checked and turns out to be true. And what we found, and we published these results in 2018, is that the general pattern across the entire history of Twitter, we built a database of many, six major fact-checking organizations took all the news stories they had fact-checked and looked systematically over the history of Twitter and found, in general, that false stories have a huge advantage. They tend to spread much faster and much further than true news. So these are some examples, which, as we started thinking about the conditions on the ground, so to speak, of being actually immersed in the actual day-to-day conversations of Twitter and other social platforms. And also, we started looking at top radio, just the different public spaces that tend to be where public dialogue plays out, that there's an awful lot of shouting at one another where people tend to hear the most extreme points of view that are reactive and often disconnected from everyday life and lead to a divisive kind of environment. And of course, the business models of both mainstream media and social media tend to reinforce and reward extreme and reactive content. And so we became interested in thinking about designing new spaces, even if they don't have the scale, of course, of the dominant platforms, which were designed for listening where nuanced, reflective conversations grounded in our everyday lives were more likely to flourish where you would end up with a connective rather and divisive kind of spaces. And that's what we mean by the word constructive in the name of our center. These various terms kind of get at what we're interested in. I mean, there's sort of three main ideas that kind of flow through many of the projects that are now, that we are working on in the center. One is to find a rebalancing of the role of machines so that there is more agency and control put back in the hands of humans where AI and machine learning we see as power tools that can be used to equip people as opposed to what's happened with a lot of the social spaces today where the AI, we think, has too much agency over what we see, what we're exposed to and who we connect with. So it's kind of rebalancing. Second is to think about rather than the online spaces as being sort of alternative realities to find ways to weave them back together with in-person connection. And even though the internet has this incredible global scale and speed and reach, can we leverage that while creating solutions and spaces that are tuned to locally differentiated needs? As Wes mentioned, we have an engineering team that Wes leads together with a group of researchers, graduate students and faculty and postdocs and so forth. What's interesting is that we can translate research concepts and kind of exploratory research into relatively robust tools that are engineered so that we can actually deploy them with partners outside of MIT and run scalable field pilots. So it's this combination of three different skill sets that makes the center, we think, quite unique within at least the academic arena. And with all this, what are we trying to build? We're trying to prototype a better future for communication, media, and social networks. And in terms of the human impact that we come back to over and over, it's to surface under her voices, to bridge divides across different perspectives, and to put all this to work in systems that ultimately we hope can build social trust. So there's many projects we're working on, and I'm not gonna go through them in detail, I'm just gonna highlight one for you in a minute. But we are thinking about projects and sort of tools and designs designed for families in the civic space, which is what I'll highlight for you. We've started doing some work in thinking about how tools can be used to inform public health communications. We'll continue to do work in analyzing and modeling media dynamics, and we started to take interest in some of the work that's coming from actually our civic space to look at what we could also do within organizations, companies, for example, where very similar kinds of social conflict and fragmentation are showing up. So just two, actually, I guess I have a one slide summary, which I thought you might find interesting. We have a team, and Wes is actually a contributor to this team, that's in the research design phase of thinking about and designing concepts for a pro-social media platform that's designed for early teens, explicitly designed to promote positive identity development and self-confidence during the most formative and vulnerable teen years. So we've been working with teens and designing reps, and plan to launch a kind of initial pilot platform early next year. So that's sort of in the works. The second example is what we call the Local Voices Network, and here I'll take you in a little bit more detail. And the goal here is to surface under her voices and experiences for listening, learning, and constructive action. This is actually a project which is being operated by a non-profit, a 501C3 non-profit called Portico, I'm co-founder and chair of Portico, which grew out of the predecessor to the center, and it works in partnership with the MIT Center to actually operate this project. And so we are the research partner and Portico operates the network and supports partner projects. So the idea was to merge facilitated dialogue and technology to create a kind of scalable system. That's the idea of the Local Voices Network. So on the facilitation side, we've worked for several years now to develop and sort of test and iterate on the design of a conversation format, which at its heart is really designed to allow participants in small groups to share hopes and concerns. It could be in general about life in their community, or it could be about a specific topic like public safety or health, that is grounded in sharing experiences they've had that relate to either the hopes or the concerns. And I'll play you some examples in a minute. We also have designed a piece of hardware, we call this the digital hearth. It's a battery operated self-contained device that is designed to actually record a high quality audio recording of small group conversations and also, so there's a primary layer mic, there's a microphone array and there's also a high quality speaker so you can play back excerpts of voice from other conversations. And this is a controller where the facilitator will pull them up on the computer and control them hard. So we launched in 2019 in Madison, Wisconsin with a key collaborator, is Professor Kathy Kramer, who's a political scientist who's done some very innovative work in developing deep listening methods for understanding public opinion. So she partnered with us and helped shape the local voice network design and then launched it in her own town of Madison where we started inviting community partners into training sessions to both learn how to use the technology and also to learn how to, that the facilitation sort of structure of these conversations and then to invite people from their networks into small group conversations which could be recorded. And of course, with the pandemic, we were unable to do the physical meetings so we switched to Zoom. In general, when you talk to people after they participate in one of these conversations, they tend to report that there was really a space for deep listening where they could listen and learn and speak and be heard where there was a space for nuanced conversation. In general, people sharing experience is not a place to come and to argue your opinion on something but rather to share your lived experience as it relates to certain topics. I thought I'd just show you a little bit of just some of the tooling. So what happens once a conversation is uploaded either from that device or from a Zoom recording is it goes into this web-based tool and there are different collections like that different partners collect. I'll just go to the Madison pilot as an example. So we can see all the conversations that have been uploaded in this collection from Madison. If you click on into one of these conversations, it takes us into a detailed viewer for that conversation. You can see different topics, housing, nurses, homelessness, prison that came up, racism. If I click on one of these, I can drill in and listen to the specific parts of the conversation. I've been dealing with racism for the longest and I turned the other team. So we can navigate into conversations and then in any particular conversation, we can, if there's a piece of the conversation that we want to lift up and share or you sense making, you can just select this part of text. The speech is automatically selected with it and then we can add a note, add tags and this lifts up and stores this piece of audio. It keeps the pointer back to the full conversation but creates a new piece of data that can then be used upstream. I'll show you a second one example of what that looks like. One other, just a couple of other things. There's also an automatically generated index. So if you're interested in housing, we can see different terms that our system has clustered that are related to housing and then we can go into specific conversations where there is particularly, here's of course, these are housing advocates. So it was actually a topical conversation and two topics of interest. There's also a search engine so we can do just keyword or sentence based search. So those are some of the tools and let me just put these pieces together now. Just give you a sketch of one recent case study that we are quite excited about. This was actually in Madison. So just to show you how these pieces come together. Once a small group dialogue has been recorded, it goes into the indexed search engine and so do a couple of different ways that you can then access the conversational data. One of the things that the hearth or through Zoom, a host of a future conversation can actually take one of those highlighted pieces of audio which might be a story for example that someone has shared of what they've experienced. Perhaps in a context that this group has never experienced personally and the host can bring that voice in, introduce the person and then play the audio. We call this cross pollination and then the host, then people in this conversation can respond to that person and in this way we end up with a conversation network. All of these conversations are being archived and we have, and I'll talk about data permissions at the end, but with access to this collection of conversations what we can do is by taking, this is again just depicting the speaking pattern of one conversation, we can take a collection of conversations and do pattern analysis which is what we've started experimenting with. So you might for example, and the example I'll show you relates to public safety and policing. So say we identify all of the segments of audio that are related to public safety and policing and then we might up wait and prioritize the subset where it's people who are sharing firsthand experiences that they themselves have experienced, not something they saw or heard about but they themselves experienced. And then prioritize those pieces of audio, we can sort them using our tools powered by people into themes such as power, trust and fear. And Caesar, can you give me some advice? Have I gone too long or do we have time for a three minute audio sample? It's done, I think you just do a little bit of the audio. I think the more important thing is just to report out because you're gonna do the piece from the police, right? Yeah, right. So, okay, let me just play a little bit of the audio and then I'll skip, so Audrey just, you can get a sense of the texture of the speech and then I'll show you what we did with this. It's hard to get away from how powerful, you know, the institution and the badge and having a gun is and how much that emboldens individuals. Well, I mean, growing up, one of the first values and principles that I was taught was to never trust the police in any situation of such a sense. And then that was kind of proven to me around the age of 12 and 13 when I saw a family member be shot in the back eight times. So what we then did for this particular case is we had an opportunity to work with the committee that was charged with hiring the police chief in Madison. There was under a racially charged situation, the previous police chief very suddenly quit his job, wrote a blog post on a Sunday night that I'm not coming in Monday morning. And so the Madison Police and Fire Commission reached out and said they were interested in running a different kind of search process and they wanted to hear from community voices to get input into the process. And we were already, we had already started this pilot So we did this analysis and for each of the themes where these purple blocks again, represent multiple different stories from different conversations that all fit a certain pattern, which we summarized each pattern and translated them into a recommended question that could be asked to the candidates for police chief. So for example, it seems that police fear some of the communities they work in and the communities fear the police in return, what fears have you observed in the communities you've pleased them. So what Cesar referred to, Audrey, is we created a report that actually went through each different theme, provided us a text summary where the words and phrases of the text summary actually pointed to examples of the actual audio snippets. And then next to the summary, we provided one or two recommended questions. So there was a report that had these multiple themes. What actually happened was that the police and fire commission chose several of our questions and explained beforehand where the questions came from that they were rooted in conversations from the community. And they used those questions in a series of public interviews with the final four candidates. And in fact, Sean Barnes, who was shown here during one of these public interviews was ultimately selected and sworn in as the new police chief earlier this year. So just to summarize, the idea here was that this very important and publicly visible point in the process of selecting a new leader, the questions were actually grounded in Community Voice. And because this tool that I showed briefly is available for other members of the Madison Public to access, it creates a transparent kind of extension to the normal public square that is more inclusive because a lot of people who would not feel comfortable going to the town square may feel more comfortable going to these small group conversations, knowing that they're being recorded and that there will be a durable voice record just like this conversation today. And that by being able to see where your voice goes and know that in this case, the interviews are being run by, in response to these patterns, it creates that kind of accountability, a kind of mutual visibility that we think is really important. And so let me stop sharing. That was the example that I wanted to... This is Drew, so thank you for listening. We, I'll just note one thing that we're in the midst now of discussing internally, which is the data model. What we're thinking about is having a model of a kind of a data trust where the nonprofit cortico serves as the data, the trustee of the data, and each of the different partners that gather people in their community own that collection of conversations and under certain conditions, other partners that are gathering similar kinds of conversation can all listen to one another. And as for that whole system is, this sort of steward of the data is the nonprofit. That's sort of a model that we're trying to clean up and put into place. So that's a sketch of our center and one of the major projects. And I will just mention that just by total coincidence, just earlier today, we hosted a talk and then had an in-depth discussion with Colin McGill. And I mean, those are two conversations that started at different moments with different people in our team. I mean, I had reached out to Colin and such a strange coincidence that both of these conversations are happening today. But we have been exploring, Colin and I, possible connections between Local Voices Network and Polis where it might be that the cold start problem of where do you get seed opinion statements? It's very interesting to us to think about these participant-driven, small group conversations as a source for potentially getting statements that can seed Polis. And actually, we just after the conversation today with Colin, I had a conversation with one of our collaborators, Kathy Kramer, and she also thought that once Polis is able to help identify some statements that may have either be divisive or connective, those could be a very interesting prompt for LVN conversations to, you know, sort of the kind of different ways these could be connected. So we're taking real interest in Polis and also very interested in, of course, your binary use of Polis and what you've done in the Taiwan. So I think I could keep going and going, but I better stop talking. That's fine, it's fine. I enjoy listening. That's what we're here for. Thank you. I have three very quick kind of clarifying questions. One is that does it depend on the equipment that you mentioned, or is it okay if we, for example, just import this GC conversation into the system? We just need an audio recording. The hardware device, because it has a microphone array, makes it easier to separate speakers because we want to have the kind of association of speakers to the different ribbons, but it does not have to be zoomed at any high quality audio recording. Okay, so even if it's in a single track, you just do a speaker recognition to tease it apart. But of course, if it starts multi-track, like some podcast recording software to begin with, then of course that simplifies your work. That's much, much better. Okay, okay. Yeah, because that's what we've been experimenting with. So the second thing. So you mentioned that you would like to work with the nonprofit as a quote unquote data trust where they do cooperative stewardship. Does it mean that the software code, because like Polis it's now running at Polis the GOV, the TW, which is our own data center with the entire stack, including the data and the source code, we hire professional penetration testers and things like that. So it's just like any other open source free software labor project. So is your software designed to act like this, like in a self-hosted situation and with many forks and the upstream and things like that? At the moment now, but it's an interesting model where definitely as we're getting to know Polis and Colm just learning about all of that. Currently Wes, I think it was a fair to say Wes was the head of engineering at Portico before coming over to the center. So it was quite familiar with the slides. Wes, any comments on that? I think you're muted. Yeah, we need to unmute, yes. We use a fairly standard containerization and deployment strategy. So it wouldn't be a whole lot of effort, I think to start up another instance of the site and the infrastructure. I mean, it's not a trivial amount of work, but it's doable. But it's not designed to work in like a federated way where we then join audio as a higher note or something, it's just, it would be separate. No, I mean, the project management side because when Polis open sourced, the kind of first major contribution, I think was from the Canadian government where they did this automotive English French because for federal conversations, they're required to be bilingual. And then we adopted that and then contributed to like the English Mandarin, auto translation stuff and so on. So my point is, as long as it's open sourced to the data steward, the data steward it will be motivated to do customizations to usually begin with maybe in your context English Spanish multilingualization. And so the management, the governance model changes. Previously, it's kind of a software as a, sorry, service as a software substitute, but now once it becomes self-hosted and people start modifying it, you'll be working with a more kind of co-governing relationship of the project itself. Because that's kind of a logical consequence and is it aligned with your values? Yeah, I think the product engineering team is receptive to making the code open source. I mean, yeah, I don't think there would be any objections. I think to this day, it's really been a matter of like prioritizing the work stream and making it open source. Sure, of course, of course. Yeah, because once it's open source, it doesn't have to go to GitHub immediately, right? Open source only means when you're, when the people using your software ask you nicely, you provide a copy of the source code. That's the original definition. So, right. Well, ideally, we would make at least some kind of a commitment to help people fix problems and stuff. Yeah, that's right. So, and once you do that, we'll be happy to provide the same service we did provide to Sandstorm, Polis, and so on, which is world-class penetration testers to make sure that it's cybersecurity-wise, it's hardened. And so that's something we can do right away once it's open source, not on GitHub, like open source by commitment, okay? And the third thing is a very minor thing. I witnessed that in the deliberative space, around two thirds of people say hopes and fears. One third of people say hopes and concerns. Is there a conscious choice that you went with the hopes and concerns? Oh, well, there's an interesting DNA to the conversation structure. There's a nonprofit group called the Public Conversations Project that started maybe 30 years ago. Caesar, is that true? No. You know the history better than me, with kind of a family counseling kind of approach to having difficult conversations. And that group helped design the first iteration of our conversation. It's now essential partners. And then eventually the executive director of essential partners became the CEO of Quartico, the nonprofit. And so we did that work with them. I believe that the basic framing of hopes and concerns either came from them or from Cathy Cramer, one or the other. Yeah, so your question is why concern versus fear? Right, because concerns is more like that we're co-creators here, right? It's on the table. Hopes and fears is like there's a power imbalance and we're here to try to write that power imbalance. So it's rather a different approach to study agenda. Yeah, that's it. Okay, no, just a random question. Okay. Yeah, so I was just thinking in another project we're testing out with us with a technique I've been using, when we start the conversations, we have a framing. So the framing we're using this one is we ask people what's their question about the future of America. And people start the conversation and see what their question is. And then the next question is, what's your experience in your life that actually got you to that question? And we use that as a vehicle for opening up kind of the conversation. So it's not waiting either way, it's just like, you know, you have something you have a question about and you have some experiences and then what we try to do in some of these efforts is connect people who have similar questions to share their experience so they can have a broader view of how the thing they're interested in actually its experience in the broader has really different entry points to it. Mm-hmm, mm-hmm, mm-hmm. Okay, that's very clear, thank you. Yeah. Cool. Anything else you would like to bring up? I asked my clarifying questions. Oh, you moved it. I was just gonna say that I'm definitely curious about your experience with Polis. We've read some of the case studies of how it's been used in Taiwan and we heard quite a bit more about the tool and plans from Colin. I was curious, or kind of current status, is it a remate in you as a tool? Yeah, it's part of our public infrastructure, right? Polis.gov.tw means it's a kind of permanent part of the governance mechanism, right? We run the penetration testing, we work with Colin to fix cybersecurity issues, we wrote our own terms of service, which I believe is already deployed now. And so in all senses, we are now a kind of co-governing node of the Polis ecosystem because everything that our public service needs to run Polis, they can just do so, just like launching a, I don't know, Google Forms survey or a survey cake or something, but understanding that a cybersecurity situation is better, right? So it's now a kind of general purpose tool for all the public service to use. And there's a public instance also at polis.tw, where the civil society investigative journalist and so on also can use. Of course, it doesn't carry the same .gov.tw like government initiative status, but maintenance and code-wise is the same cybersecurity-hardened coding infrastructure. And in your experience, has there been certain patterns of where, what kind of situations it is most used for or helpful and also where you have observed the biggest pain points? Well, what's difficult about using it? I mean, we ask, call them the same questions and I can share some of the things we learned, but I was curious what you had. I mean, we have a national level regulation that says when to use tools like Polis. And so, and we have teams in each ministry and competent authority that says that the training is, which training is required to run such issues. So I'll be posting that into the chat. I believe there is a entire website about principles of processing collaborative topics. And so things that have complex stakes, diverging views of multiple stakeholders and enthusiastic publish participation that results in the need of interdepartmental collaboration. These are where Polis shines. I'm posting the link here. On that website you can, under the Collaborative Meetings menu, see the process toolkits, the guidelines, all the past 100 or so collaborative meetings, the directions, the national regulations that enable this 100 or so person collaborative facilitator in the government structure. So I'll not be reading it out loud here, but we do have a pretty well-tested process in the past five years that runs these things. You know, what's fascinating for me about this kind of here is the different systems for actually integrating something into a governmental infrastructure. It's just very different in our society, right? I mean, for us, some of our Polis, you almost have to think about how to grab it in civil society as a mechanism as opposed to actually a tool for government, because our processes are so different. And that's not a good thing, it's what I'm saying, it's not a good thing. Yeah, but our GovTech is actually civic tech just implemented by government contractors. So that's the fundamental difference. In the US, I understand the USDS is like us, right? It's the GovTech and the 18F was more kind of connected to the covert America, covert all and so on. But I don't think there is a single case where the civic tech side wrote this specification and did a reverse procurement where the government has to implement it. So it only flows this way and not that way is what I mean. But in our case, it's almost always flows this way. I'm pasting a link in a recent SMS based check and trace system that was co-developed that way as well on our blog. So I think that's the main kind of infrastructural difference between our polities, which is why I'm very excited to try out the kind of quality enhancing topic to address your question about the pain points of Polis is that when something is very enthusiastic publicly and we get like 300, 500 statements, like each person only goes through some of them and currently it's just showing things randomly. And so people's average experience degrades slightly the more statements they're in. But if we do have this kind of quality highlighting mechanism, then we actually uplift the experience every time that's the rough consensus or good enough consensus is reflected back into the Polis system. So Polis become the connector between the in-person conversations, just like machines should be the kind of in-between of humans. So it's not about placing machines in the place of facilitators, but placing machines across time and space for different facilitators to work together. What do you mean by lifting up high quality and I don't understand that part? Sure. So part of our use in V-Taiwan in Polis is to invite the people who have received a high resonance across different groups in Polis and invite them to the meeting place, face-to-face in person, or at least through high quality broadband video conference to share the full context that led to them, their kind of eclectic interpreting statements that get so much support. So in a sense, this is more like trusting the persons who posted these ideas that resonate with all to kind of serve as advocates to contextualize this. But your system, as I understand, flips this around. In small group conversations, I can talk for hours, but the moments where I behave in a pro-social way can be highlighted and say this moment of yours resonates with other moments of these people. And so it highlights the statements, the commonhood of the statements like a common, how might we question? That could be fed into the Polis conversation so that in the Polis conversation, we start with the arguments that already have good enough resonance and then the Polis situation will begin to innovate even more because we spend less time to go through the parts that checks the kind of missing contacts, missing pieces and so on. We start with kind of, by definition, more resonating statements. And then once we get the good enough consensus around that conversation, then we set that as a topic for more small group conversations to work with. So in the beginning of each Polis conversation, people don't feel that they are spending a lot of time to weave through the points that has been iterated before. Got it, got it. Yes, that's very much the kind of concept that we discussed with Colin. I had to discuss with Colin earlier today and he's very interested in exploring as well. Do you have a sense of what kind of people today, what are the characteristics of citizens who are more likely to engage with? Sure, people who are 16 or 17 years old or 60 or 70 years old, these are the two age groups that are the most active on our participation platforms. A, they have more time on their hands and B, they care about the next generation, kind of by definition. And anything regarding, are people who, interesting, I was just thinking about how else to kind of other dimensions, major dimensions, like education, gender, but actually what I started thinking about... We don't see much of that. In Taiwan, we do have universal broadband and digital competence education and healthcare as a human right. So we don't see much pattern differences between the conversations that had on the rural or urban areas. Of course, they may bring different viewpoints but the participating activity is very similar. And I would think, I've found it always fascinating in the United States, some of the patterns around the spoken word versus the written word and the relationship to level of education and literacy rates. I don't mean necessarily functional illiterate, but if you are less comfortable with the written word or the preference for radio as consumption, since everything is text based, is there any selection bias there, people who feel more versus less comfortable expressing themselves in writing? Well, there are just like in participatory budgeting process. There are people who are more effinent with text that participate on the discussion board stage, but there are people who are more comfortable with the spoken word. So they do that on the kind of collaborative meeting stage, just like the PB process. We always have a in-person plus video conference collaboration meeting stage that holds ourself to account to explore the agenda that's set by the Polish stage. So there are people who wait until that moment to join and previously they mostly just observe. Of course that's quite natural but when it comes to the kind of selecting the cases through e-petition or in Taiwan we have referenda, right? Then because really just pressed like, which is what a referenda petition as a contentment is, it is so low threshold, everybody does it anyway. So what I mean is that we create a space for the modes of conversation that are tailor made to the people who are comfortable with that conversation. So for example, we're held town halls where I travel to the more rural or remote areas and join them where they are as a facilitator. But then we video conference in the central government people but it serves explicitly as a kind of expert, consultative expert role, meaning that they don't share anything until asked but they provide the contextualizing information to the local people over a video conference without having to travel over Taiwan and so on. So we designed a space always to put inclusivity first and people who are not comfortable with some mode of conversation either just trust their friends to speak on their behalf in that mode or they wait until the part in the process where their preferred mode shines. I know we don't have much more of your time. And there are a couple of things that I can't see here that I just want to kind of test it. One is actually a question that when folks who were going to have this conversation there were a whole bunch of folks there. I want to be there, I want to be there. And the question is, can you be open to maybe some time in the fall, some time actually every open conversation? Yeah, definitely, definitely. Yeah, it would be great. We'd love to do that. And then it seems like in this conversation about a poem or something, what Congress is dealing with, what you have in place here, actually maybe a kind of three way way to work together in the pen, thinking about this information or something like that will be in the poems in some place that takes to that or we'll see or something, maybe the meeting viewers or ours. I don't know if it will or not, but maybe it's also a place where we can kind of test out some, possibly working together. Yeah, definitely, definitely. Because now I understand we can retroactively feed existing audio recordings as long as it's quite high quality into the system. Instead of running new consultations, one of the first things we can do if we have access to the system is just to put the 100 or so collaborative meeting which are all recorded in rather high quality and see if it works. And then, because the majority of our conversation was in Mandarin, we will also be testing the multilingualization and internationalization capabilities. Very interesting. Who there works, anything else you want to say as well? It's just a small part of my question there. So these conversations that you're referring to, on average how many participants are there? So the small groups are always like 10 persons per table ish with three to maybe five tables. So a small room basically, but for each collaborative meeting topic we may hold multiple such rooms and sometimes interconnected in the kind of informed stage. And then we break out into multiple rooms and we almost always have a video recording that has separate channels. And sometimes we also live stream it and then their live streaming may be played in some other room which then have their own conversation. That's done as well. Thank you very much for taking the time to be with us this evening. And I think this is just the first of another conversation and we'll reach back out to Francis about the exclusions in the fall. Yeah. That would be great. Dan, anything you want to say to the closing? Just thank you for your time. Fascinating to hear about your incredible work and I really am excited to welcome you into the MIT community sometime in the fall to share this for the larger group. Thank you. And I notice on the chat there's a question about digital divide and I will just very quickly say that in Taiwan because Universal Broadband is a human rights we don't have that much of a problem there. I understand theoretically in politics with a digital divide it will be democratically illegitimate to run this as part of a democratic institution but we simply don't have that problem there. And in the 20 or so percent of people who've never installed an app for public tool purposes that's exactly why we made sure that you can use toll-free numbers as a mass-based system showing up in person and things like that instead of interacting through specific apps. So I personally have flip phones running the KaiOS. So anything that phones can run is inclusive but bandwidth is unlimited data plan for just 16 US dollars per month. So we're pretty good there. Yeah. Wow. You guys see. Again, thanks so much minister and fastest thank you for everything and talk to you soon. Thank you. Live long and prosper. Bye. Take care. Bye. Bye.