 that will be reported out into public. So I've enjoyed very much the series of conversations we've had. So today we've pulled together people from various, who have worked on various government services. And I think maybe I could get each of them to introduce themselves. Maybe Audrey, can I start with you? Yeah, certainly. And officially, I only began recording like now. So the after-record conversation wasn't part of the video. Yeah, I'm Audrey Talent, I'm a stational minister in charge of social innovation, open government, and youth engagement, which really is the same thing anyway to me. And I'm really happy to be here. Thank you. And Beth? Hi, Beth Novak. Currently serving as the chief innovation officer for the state of New Jersey was previously the deputy chief technology officer and the head of open government for the Obama administration. I've also advised both the German chancellor, Angela Merkel, and worked for David Cameron, prime minister of the UK. And in my capacity as head of the GovLab, I work with governments around the world on digital transformation and governance innovation. Thanks. Thanks. And Hal? Oh, you're on mute. Sorry. Hi, good morning. I'm Hal Sekhi from Code for Japan. I'm a founder of Code for Japan. And Code for Japan is a non-profit organization for facilitating the speak tech activities in Japan. And also, I'm working for the digital agency, Japan, as a project manager of Code for Japan, project manager of the speak tech activities. Thank you. Thank you. And Miyata sensei, who's my co-chair, maybe you can introduce yourself too. Yeah. My name is Miyata. My major is data science, as well as a scientific method to improve the real world. And let us quick introduce the concept of the co-innovation architecture design conference. We like to update the concept of the SMAT city. So what is SMAT? The SMAT is for the industry or for the big stakeholder. We would like to change the concept for the people centric. And also the aims is not for the such kind of the economies or the profits, just for the well-being, sustainable future and the health, well-being and diversity inclusion. We set up our aims like such kind of upper layer and the collaborate together. And the concept is not only the physical world, but also the utilizing the metaverse on the such kind of the virtual community and network and the set. So we would like to develop the multi-layered democracy through such kind of the new co-innovation architecture method. Thank you very much. Thank you, Miyata sensei. And I think this is maybe a good place to kick off. I'll pick certain topics, but again, feel free to sort of take the conversation into slightly different areas. But first of all, I think the title is important. And this is something that Miyata-san and I and others have kind of worked on to develop. And it's so the idea of co-innovation and architecture and design are all pretty important words for us. And I think co-innovation I think is key because up till now government services have been mostly centrally planned. And as one way of saying it, it's been very supply side oriented. It's kind of like I'm designed by for the people who have the power. And the idea of co-innovation, which is could be co-design or people centric design is the government has started to say it, which is good, but it's not clear that we really know how to do it. And so that would be one of the really important topics to talk about. And I guess we could kind of start there if people are okay, I mean, I think, I know Audrey, you've shared with me in the past, a lot of the ways that you've engaged citizens in design, but maybe Sik-san, so Sik-san runs is in a really interesting position, I think, which is code for Japan is, I think one of the largest groups that do community oriented hackathons and design, but he's also right in the middle of the digital agency working on projects and materials, the term co-design, but I'm not sure, maybe you can report a little bit on what's the state of sort of design practices and co-design in Japan. And then maybe we can hear from Beth and Audrey examples and advice on how we might make it better. Okay, so let me share quickly about the situation of the co-innovation between the citizen and the government in Japan. And as a code for Japan founded in 2013, and gradually the student side of the student side communities are growing, we have around 19 local code for communities and they are working with the local government level. And there are some good examples about the co-innovative project and we created some tools as an open source. And but the recent tree, I feel that there are, many of the projects are lead by the Burantali groups and the government doesn't know how to collaborate with them as a foundation level. And also that we have some difficulty about making good policies together. Still the policy level, the planning is led by the top level of decision making process and a student are not much involved in that process. So that we started a project called make our city project and provide some tools like decedent. It is open source civic engagement tools born in Barcelona. But still that activities are weak and no budget, budgeting from the many of government. Yeah, we are still challenging to create policies together. And Audrey, do you wanna go next on some of the things that have been worked for you and getting citizens to participate in design and policy as well? Yeah, certainly. First of all, I think I thank you for this new branding of co-innovation. It's a brand new SEO term that we're free to add no more meanings to. And in a kind of previous iteration when it was still called social innovation, agile co-governance and so on, I've had a conversation with many folks. I just paste it to the chat. It's published on the NBL magazine in both languages, English and Japanese that talked broadly about the same ideas. So I'll just highlight two points. The first is that the civil service, the public civil servants in the front line in those local areas, they probably already have all the required innovation figured out. It's just they don't have the political support or the budgets or the public communication expertise or whatever to realize those innovations. So to me, one of the most important thing in regional rejuvenation or revitalization is to get them a safe space to get those ideas, let's say escalated or amplified on the national forum so that they get the mandate that they would need to convince their mayors. This has been the most important thing that we've seen in contact tracing, in vaccination, in mass distribution and so on. The same patterns repeat itself so much so that we organized the presidential hackathon around this very principle of public service co-innovation. So that's my first point. My second point is that when people see the good idea, the good models and so on, they're going to ask, so how are we going to fund those things? And in Taiwan, we're basically using this idea that we call pay for success. I know it has at least seven different names, like retroactive public good funding or whatever, right? There's somebody who says kind of the debt, but it's not really a debt, it's an investment and so on. That is co-created, co-designed, and we basically ask the private sector to help fund these in the expectation of a contract of a return by the government when these innovations actually create social values that could be at least monetized in such a way. Or it's not really monetized, it's dollarized, right, as to you can put a social return of investment on it. Now the crux of the matter is to get a people-public-private partnership where the people set a norm in which that the public sector ask the private sector to put a kind of dollar amount on the time it saved, on the budget it saved and so on, so that we can actually pay out and we structure it like an award, not a particular grant, and that's the structure that we've been engaging the youth in making sure that they participate in the regional rejuvenation attempts without worrying too much about the initial capital investment because the traditional VCs, unless we design something like pay for success would not put money into it. So that's just my two very brief points. Thanks, Audrey, and I might have more questions about that. I will follow up, I'll read this first and then follow up. But Beth, I don't know if you have experience in this space that you could share for us. I've written three books on the topic, Joey, so plenty of experience. And we're tomorrow running a co-design project that we've done most recently in five cities in Mexico, previously in five cities in Africa. Tomorrow I'm running the same project in California to co-design solutions to urban challenges between government and residents. And I'm happy to share more details on this, but let me on the model that we use for doing this kind of co-design work. But let me share three kind of broader points here. One is that I think it's very important to recognize that there are different types of co-design or collective intelligence or engagement or whatever term we wanna use. And to be very clear about when we're trying to engage in human-centered design versus crowdsourcing, versus open innovation, versus collaboration. So it's, I think, just crucial to be clear on whether the goal is to try to identify a problem or to solve a problem together or to implement a solution to a problem or to evaluate collectively what is or isn't working. For these different stages of policy implementation or project management, it's just useful to have the right platform and it becomes very easy to use the wrong platform for the wrong purpose or the wrong process for the wrong purpose. So I think starting from the perspective of what are you trying to accomplish is incredibly useful as just a basic idea for knowing what kind of a co-design process to implement, number one. Number two, I think it's recognizing, as you explained at the outset, Joey, that a lot of the discussions about co-design are still what I would call lip service. People are talking about it, but they don't have experience doing it. They haven't been trained in how to do it in university or in graduate school. It's not part of the job description. So if you're coming out of a tech culture and you know what a hackathon is, it's one thing, but people in government are not trained in this way. So I think training is extraordinarily important. That's where some of Audrey's work around the participation officers network and teaching people how to co-design and co-create is just so crucial. So I wouldn't underestimate the importance of talent and capacity building to be able to do co-design, just as we know that you need some training in data science to be able to use data and open data, you need training to be able to engage in co-design. So I'll stop just by putting a link in the chat, not to any of the things I mentioned, but to something else, which is a series of global case studies of governments co-designing with residents around the world. And then a guide, a short and longer paper that we did with Nesta on sort of what are the lessons learned from that. So if you click on the report, you'll see there's a short guide. We have everything summarized in one picture, but there's a lot of really interesting global cases around how people are doing different things like defining a problem together or solving a problem together. So let me stop with that. Excellent, thank you. And just as a process point, I'll inject that I'm gonna do a joey-styled conversation-style discussion. So if anybody has anything to say, there's a little smiley face and there's a thing to raise your hand. So just raise your hand if you have something to say, otherwise we're just gonna go sort of free association because one of the topics I think that I wanted to cover as well that Beth started talking about, which is capacity building and training and learning. And I think that one of the problems that I see quite often in addition to just lip service is very well-meaning decisions and discussions that this is what needs to happen without any real mechanism of how it's going to happen. And so a lot of these reports, including the one that we're gonna be feeding into, I can imagine we'll have a lot of very visionary ideas including co-design without any real who's gonna do it and how they're gonna do it. And one of the things that I see is capacity building is always the second thing after, okay, here's what we need to do. But as you say, I mean, I think there's like traditional capacity building is just feeding people information that then they consume. But I don't think that's the way that people learn how to do things like co-design. And I'm curious sort of, I know that, and by the way, sending us links is great because these are all things that we can consume and we'll put them in the report. But if you have either links or ideas on what have been effective ways, because I think of things like with pair programming I always think is wonderful because at the end of several months you have twice as many people who know how to do it. You can keep adding people to this ongoing thing. But I wonder for co-design and again, given that, as you said, there are many different types of co-design and co-innovation, but just for any category, have you found what is the most effective way to increase capacity both in government as well as in local regions? I don't know, please go ahead if anybody wants to jump in. Yeah, can I go, how was that? Yeah, so, yeah, I agree that the working together is really important to cultivate the trust with the people and the government officials. And I suggest the, I often suggest the local government to have a field to work together with the students and for example, Tokyo Metropolitan Government recently organized hackathon and invite local officials, local government officials and developers and also students. And I see the people inside the government realize all students is not, some of the students are very much, has intends to work and make the city better place. And after they realized that kind of the correlations, their mind became open. And I think it's important to create such kind of the field to work with the students and not only just discussion. I mean, Beth, I don't know if you have any thoughts on the role of hackathons. This is something that Code for Japan does really well and Audrey as well. I mean, and I guess maybe if I interpret a little bit of what Hal is saying is, first of all, building the trust that you developed when you're actually working together on something I think is a very good first step before you get to more sophisticated things like platforms and project management systems. But I'm curious whether you have any thoughts on where you go, how to go from hackathon or first of all, how to do a good hackathon and then how to go from hackathon to more, let's call them core services that we might be able to deploy. I don't know if you have any. I think the lesson for hackathons or for any kind of engagement is the same which is you have to make it relevant. What works so well about, I'm gonna sing Audrey's praises because she's here and it's fun to do and better than talking only about myself. I'll give some other examples too since Audrey can talk best about her stuff but the great part about what Audrey does is there's an outcome to it. It's not citizen engagement or participation for its own sake. The fact that Taiwan has the example of saying we've crafted 26 pieces of national legislation as a result of co-design is much more powerful, frankly, than all of these conversations about French citizens assemblies where none of the recommendations go anywhere. And similarly, what makes for a bad hackathon is a bunch of people making apps that nobody's going to use. So I think the important thing is not to wind up the machinery of engagement or open innovation, co-design, co-creation without the ability to explain how it's useful. It doesn't mean you have to promise to use everything people tell you. You just have to be able to explain the purpose. So tomorrow I just came from a two hour meeting to plan, we have 30 groups in the room tomorrow and we're asking them to spend two days with us designing an engagement. And we have to, all of the work goes into explaining to them, preparing how to explain to people why we're going to treat their time with care and with respect and what the outcomes will be. So it's about clarity of instructions and directions, clarity of workflow and having thought through the process ahead of time. And that, to come back to the original question is why the training is so important because you need to teach people how to do that and how to articulate that kind of an agenda. Even or especially with something more basic like human centered design, where I'm just asking you maybe to test a website for me or asking you some questions so I can design a better website or service. You know, the best practice is the ability to explain to somebody how their input is going to be used and potentially we're needed to pay them for their time but that's another topic. And to be sure that you're inviting the right people which we can get into. But I think long story short, make it relevant. Miyazaki sensei. Thank you very much. I deeply respect your project, sexual pay for success or the hackathon and empowerment people. It's very important I think. And on the other hand, we like, we better also better to develop the platform to utilizing such to create the share, co-create shared value. So the best already say it's like such kind of purpose is very important. And so sometimes we play on the Google platform or the meta platforms, but such kind of economic driven platform algorithms. Sometime the purpose is for the money and so stay longer, make the much money. But we sometimes discuss with them about how we conquer that. So what is the social good? And so sometimes we like to share the data and also make co-create the algorithm for the sustainable future or the webbing. So the economic purpose is the second one. So such kind of the purpose setting platform, co-innovation is an important topic for the next step I think. Actually, can I toss this to you Audrey? Because in our podcast discussion, you made a very interesting point about how purpose driven education is actually more like is upstream in my view than project based learning. And you were describing how children in schools start with a purpose and then develop a project and then learn to code. And I thought that path was very interesting and I'm very curious how you think about purpose in this sort of context that we're talking and maybe capacity building too. Yeah, certainly. So I'll use a very concrete example of next year, actually this year, this year is presidential hackathon. Presidential hackathon 2022 is funded by the National Science Council, similar to the National Science Foundation. The actual implementer is the Ministry of Interior. And it's only on open data and data pipelines and privacy enhancing technology and so on does the digital ministry come to play. So this is a very clear kind of strata, right? The National Science Council does this shared goals definition, co-definition. It's firmly on the problem identification phase. The Ministry of Interior, because the theme of the hackathon is, I don't know how to translate, but anyway, it's a co-habitable balanced Taiwan. So basically regional rejuvenation, same as your topic. So we have two phases. One is multi-month, what we call the idea song where we engage the artists, the poets, the movie makers, the game makers or whatever. And their job is not to produce any code, although I think some code will still be involved, but rather to clarify the vision and to put into visual or interactive arts what people's collaborative wish are. And we reward the best articulated visions and then the code development and design proof of concept pods take those visions because at that point, the vision is already kind of president certified. They know that they will get funding that puts into the National Science Foundation, National Science Council's way of kind of emerging tech funding, innovation funding track. So by that time, they already know that there will be budget allocated to this, whether it's telemedicine or teleeducation or things like that. And then when it concerns private data and things like that, well then the digital ministry comes to play, but not before. The most important thing is not before because otherwise basically solutionism enters the space and then smart citizens become trapped in smart cities and they become less smart citizens, right? So that's the general flow of work. Now in day to day detailed terms, this can work because we have well trained participation officer teams. I've pasted more than 100 different collaboration meetings that we've done together and they always again begin by a wish proposed by a citizen, but of course local public servants are also citizens. They propose many interesting things. And the pain points, the problem is often government cost, right? Whether it's a bad tax filing system, whether it's blocking mountaineers from entering the indigenous spaces. For good reason I'm sure, but not well explained or very bad ocean policy that makes it very difficult for people to find what activity they can do or whatever. They are government cost problems, which means that the government can solve them. And then when we enter the collaboration meetings, the breakout groups are facilitated not by the participation officer of the ministry involved, but rather by totally unrelated ministries. And this is my design. So when we co-created tax filing together, maybe the ocean guard, the coastal guard hosts the breakout group. And but when we talk about ocean policy, maybe it's the tax agency's participation officer. And the reason why is that the coastal guard also files their own tax and suffers from it. The tax agency officer also like surfing, right? And fishing and things like that. So the citizens when they enter this workshop with both online and face-to-face, they feel that their breakout group leader, A, knows a lot about public service and B is actually on citizen side. And then that's a very empowering move because then those participation officers really feel that they have solved the problems by not breaking out of their own silos but starting outside of those silos. And so I think that's one of the mentalities that could really be fostered very easily if you just design your co-creation groups by facilitating with senior or at least mid-level public officials who are well outside of the silo but themselves represents the same values that a local citizen has. And this is a question both to you and to Beth, but you were saying the training of these participation officers and it sounds like you have a design practice that's also being developed. I mean, how do you capture these, let's call them design practices and then how do you then train people? And I know Beth has been coordinating across countries. I'm curious, you know, one, how that's done and two, whether we can learn more practically how we might do this in Japan. I don't know, Audrey or Beth either. I'll, I'll do you first and then I'll go change it up. Sure, right. So yeah, we've been documenting everything, right? And that's part of the radical transparency thing. So for each and every collaboration meeting if you click into the CM side, you will see the mirror map, the mind map, the full transcript of what has transpired, sometimes a live streaming record and things like that. So basically we published the preparation stages by capturing it. Initially using like human transcribers, but nowadays just like our WebEx we're using more and more assistive intelligence in capturing those things, which is actually very easy if you invest into the right microphones, but because of the COVID this year, everybody has good microphones. So that makes things easier. So the point I'm making is that you don't need to actually consciously document it. It's just make sure that you have people who work remotely and make it a habit to start a conversation saying this will be on the record and actually put it on the record. And then afterwards it's very easy actually to use some simple machine learning to create a site like our CM site so that everybody can go back to a wildly popular citizens initiative, a petition or whatever and find out exactly what has transpired before that meeting because everything before that meeting is also documented. And the public servants, they are very intelligent people. They just learn from these materials. Of course we do organize workshops and things like that. I think having the raw materials that they can relate to is very important. Interesting, that's excellent. Beth, I don't know if you, and also how maybe after Beth, do you have any patterns that you can share that you would... As far as documenting learnings, we've been trying to, I think there's sort of two ways in which we do that. One is by having learning be a central focus it means we're constantly in that process of trying to figure out lessons learned. And we do that through, do things like writing case studies, but then we turn the case studies into podcasts and we turn them into one pagers because we know that the busy government professionals we work with don't have time or interest to read long case studies. So we try to distill things into very short forms. We're also trying to create more of a library of examples that we can use for teaching and training. So I'll put... Oops, wrong link. Let me find the link. I'll put a link to one of the earlier... We've been doing a lot of skills training for a long time, but then when I took this latest role in government, one of the first things I did, some of you have seen this, was to build a skills training platform. And so we were taking the lessons we were learning and turning them into short videos that we could then share with people. But then we turned the videos into a series of live master classes which we're now doing across states. And now we're starting a new project with UNDP to do this across countries. So we're constantly sort of taking what we're learning from practice and turning it into teaching materials. And we have a team that's just become really good at kind of writing scripts and editing them and sort of creating learning objects, if you will, out of these experiences. And so what happens is we start people with a very, very brief introduction to something like a 10-minute video about co-design. And then that gets them more interested. And then what happens is we just did, for example, an eight-part course on co-designing solutions to problems. So we kind of graduate people from a little bit of content to kind of a little bit of something in your own time, to something live and a little bit longer, to something slightly longer. And then we graduate from there to coaching, which is now that we've introduced you to some of these skills around co-design as well as uses of data, we then work with you to learn how to apply this to your own projects. So we've started doing office hours where you can come with a project you're working on and we kind of coach you through how to apply these skills to your own work. So it's the interplay between kind of doing and learning at the same time that I think is really useful, but having a team that can extract the experiences and turn it into something shareable that's very helpful. Interesting. I don't know, Sekisan, do you have any learnings from Code for Japan maybe that we can think about in government? Yeah, I think the teaching is really important. And we have kind of the workshops for the local government officials called Data Academies and GapTech Academies. And we have similar program and we have a series of workshops for the local officials and invite them and teach them and also not only teach and also we work with them and solve their local problems and because they have no time to only learn new things so that they have to solve their problems and so that we work with them and try to solve the small problem with them using the data visualization or using some RPAs or some other techniques. And I have a question to Beth and the difficult point is the public officials are too busy to learn these kind of new things and there are no fields to try what they learned. So the government give official support for the learners or some kind of mandatory things to learn new things. It's a really good point. I don't know the situation in Taiwan. I can tell you that in which I hope you'll tell us. In Singapore, learning is mandatory and there is a real culture of learning, the sense that I will fall behind, I will fall behind the private sector if I don't learn, if I don't train and learn. Training is treated as a reward. You get selected to be part of a course. It's a big honor to be picked. So it becomes something competitive. In Argentina, they give you points like frequent flyer points for courses that you take and that translates into economic development and it's into economic raises. So you don't get promoted without learning. In Germany, it's a little different in that they, it's free, it's again their requirements for training and it's part of your collective bargaining agreement as part of the public sector unions that you do get to do training. And so it's something, I think it's still a struggle and getting people to do the right training and what we lack for sure in the United States, we don't have, unlike in Singapore and Canada and some other places, we don't have a strategy or a vision. No one says in the 21st century, we think it's really important for public servants to know how to use data and how to engage with citizens or it's really important to know digital skills. In Singapore, they've said we think it's important for public servants to know how to code. I don't think that's the right thing, frankly, but I really like the fact that they picked something and that there's a vision for what to do, which we completely lack in the United States. So in the US, there's no incentive, there's no requirement, there's no vision. And so you really just have no culture of learning particularly. And then when people get busy, it's the very first thing that you stop doing is you stop upscaling, basically. All right, thank you. Have you yet? Yeah, I just pasted a link. It's a metro map for learning for public service and the social innovation line is line F, but next to it is the line G, the assistive intelligence line and there's also lines on circular economy and emerging technologies and things like that. If you click on those stops, it will show you the master classes, the workshops and things like that. We've tried to gamify it, it's a lot of fun and it also shows the synergies between those concepts and case studies and the resource that you can tap into to start your own presidential hackathon initiatives and so on. So I think making it fun is really the important thing. We're all experts here in the public service to make things fair and we're working on making it fast, but fun is something that's really required if you want the very busy public servants to feel like there's something to do with their job and as for mandatory, I think for the kind of working mid-level public servants to become managers in Taiwan, it's mandatory for them to go into a multi-month kind of problem solving workshop using these metros, lines as guides and basically as if they're on their own director general and what would you do? And so I got a lot of interviews from those people who are just becoming more top level senior managers because they have to essentially write their thesis. For example, how to make future payouts to the health insurances as fair and fun as the mask distribution or whether the national health insurance administration can serve as an internal cloud provider and so on. So these are all defined by themselves. It's not an assignment of any kind, but they have to defend a thesis in order to be promoted into senior management. Great. That's very cool. Miata-san, sensei. Thank you very much for your great practice. And so I believe your practice is great. And also the most important is just empower innovators and early adopters. So from perspective of social marketing, it's an optimal way to spread the new idea for the world. On the other hand, in Japan, especially in Japan, it's a hyper aging society already. And many people thinking about the late majority of the laggard that sometimes we should include such kind of the people. So just start cutting edge project from the innovators, early adopters. It's a perfect and essential. On the other hand, how do you include such kind of the people who don't want it to collaborate together or to participate together and or a lack of motivation to develop a new democracy? I'll take a stab at it. Oh, so that's my question. So, yeah, sorry. Yeah, okay. So I was yesterday just in Pindong and this is the mayor of the Pindong County. There's a small video down below the food. And so aside from me, Donning Indigenous Nation has stuff and cooking together and so on. What I was trying to show is that this is a primary school that a lot of the seniors in their 70s and so on, they were educated there. But because of the aging society and Pindong being the south most of Taiwan, there's no sufficient school children to sustain that school anymore. So basically we collaboratively with the Ministry of Education and Interior reopened the school so that it's a public park-ish now. So, and the point here is that the senior people, when they go back to their primary school, that in their memory, they can still see the pineapple fields are still there, everything is there. But next to the field and literally without walls, so you can't ignore that, is some agricultural helicopters, drones and that are basically the south Taiwan's main license field for license operators, pilots of agriculture related drones and they get to see the pineapple fields themselves can be painted into rainbow color or whatever color to attract tourists and things like that. So basically it put it somewhere they can't ignore. It's just their alum, their primary school. It's in one of the most central places so they just walk next to it. I think it won some design awards in architecture and by the night when they gather at just like a public park, they then are mingled automatically with the young people who then will listen to their pain points and then starts their social entrepreneurship proposals based on the seniors ideas or seniors pain points and so on. So the point is that we can't really, you know, really convince people of different generations to change their priorities, but we can do collaborative kitchens so that simply by showing up, the formed shared goals are first and foremost in people's minds instead of particular solutions. We don't actually do pedagogy in the sense of teaching people new solutions. We instead do co-learning in which that the local people just share their problems over dinner and it also makes the indigenous people and other people who are kind of strictly speaking less well represented in the city council level and things like that feel this sense of intersectionality because people younger 18 are also unrepresented and so on. So there's this natural solidarity going around when you have spaces like this and my own office in the social innovation lab is the same, right? We literally tore down the wall of the Air Force headquarters to convert it to the public park so people can come and play basketball or whatever and cook together. So in short, having something that is a public park and you can cook together and then everything else could follow. Great, great. Thank you very much. So such kind of the utilizing the multi-layered culture is I think is a brilliant idea. So we are also developing a lively and low level area. So we are just considering the moving bookshelf. So sometimes you already say full culture, sometimes old people get it together and never communicate with new commons. But when we share the food cultures, old people knows what is the best timing for the fruits and the new people know the various type of way of cooking and get it together and co-create new and the delicious ones. So such kind of the culture links diverse community and it's sometimes the nature or sometimes the traditional festivals and sometimes food culture and the music and so. Yeah, I think it is such kind of multi-layered culture is a good clue to get it together. Thank you very much. So we have a few minutes left and I wanted to make sure we didn't miss one topic that I wanted to talk about, which was open source and the role of open source in a lot of the, as a tool for a lot of the transformation. And I know, I mean, I've talked to Audrey and Hal separately about this. And Beth, I know you were, I used to call you the officer of open for the White House when you were there. But I think in the US there have been some announcements of trying to get to 20% open source. So there's sort of kind of a procurement level innovation. I was brainstorming with some of our digital agency senior people and there are a number of people who are in the digital agency who have open source projects on the side that are very useful and they're tied to government services but they're not rewarded for that at all. And so there's a couple of pieces. One, how can we encourage people to do open source and reward people for doing these side projects? And then, and also just from a policy perspective how do you integrate it? Assuming that you agree that it's important. I don't know if anyone has any, because we're trying to make this also one of the bullet items in the report that we're going to be producing. So I've just any wisdom. I know when the minister sent you a pull request they are very encouraged. That could be one of the requirements for becoming minister or senior person is you have to make a useful pull request before you get to the next phase. That's great. And thank you for the pull request to Japan, Japanese government services audience. No, no, no, there is by the way a whole of a lot of people who focus on kind of open source policy making too. And in terms of helping people to understand what a pull request is, it's not just code but actually doing policy in an open source way which may help to explain it. I think in terms of the arguments it's very helpful in terms of political leadership to understand the arguments that it's again the right thing to do are not very helpful. The fact that it's faster and it's cheaper. The fact that we built a new business services platform and we saved easily six to 12 months by stealing code from another jurisdiction and building on what they'd done. We were able to explain just how much time it saved us and therefore how much money it saved us. There's of course all the cybersecurity arguments but I think really appealing to the bottom line that you're gonna get the best in class kinds of services as a result is really, really important. But I think even more important than the open source code is really data interoperability and data sharing and the ability to design the whole government as a platform idea and being able to build core components and services and APIs on top of them using open data standards to be able to just accelerate innovation and digital transformation that I think is really is so much more important. So we've found very little resistance to the push toward open source software. What we still find a very heavy lift because it's hard to explain and it's legally difficult is the data sharing and data standard setting that we need. So we're about to do an executive order for example that I wrote not only to mandate radical interagency data sharing, but to basically put myself in charge so of any time somebody wants to build any kind of component that faces the public or interacts with the public that they have to come to us for the standards by which they do so and they have to develop an API for strategy. So basically we just kind of inserted ourselves in the middle by writing an executive order and executive order to give us the authority to do so. We're still trying to get it out the door, but I think the speed, the cost sharing and the ability to kind of engage in more agile transformation is something people are beginning to understand but probably in the interest of time easiest to point you to some other resources that I think are really useful on this topic. And then I don't know, Adri, you were, we talked a little bit about it. Aside from our school requests, I think it's also a political necessity if the municipal mayor and the central government's leader, the premier in Taiwan's case is not the same political party. Actually our Taipei mayor is a leader of an opposition party and in many cases data sharing arrangements and open source is the only way politically that they can collaborate on things like contact tracing because the great thing about open source collaboration is that it's collaborating in a dictionary sense like you don't have to kind of love your operator but you can still collaborate with them. So basically it's constructive criticism whenever the MPs of the opposition party ask, okay so why the open street maps community tells us your map of mask rationing is a terrible data bias because you assume everyone on a helicopter you only measure in KM kilometer distances between a person and a mask available but in the rural places they have to spend three hours blah blah blah and then the minister can say, you know it's an open API due to something else, right? What would you like to do instead? And then that that flips the kind of interpolation around because they have the same data in API standards as we do so if they think this is bad they must create something better and in this particular case because the MP was a VP data analytics Foxconn she actually says something better and then we implemented that on the central government in 24 hours. So basically if they have to submit freedom of information access requests then we're always at a disadvantage because we could be accused of, you know selectively publish the numbers and code and so on but because there's an existing open API pipeline that's updated every 30 seconds almost paradoxically the public servants in the central government is not liable then because we see the machines numbers the same time as the local people do and we're sorry that we didn't do it well but we are now committed to amplify your local innovations to a country scale in 24 hours and so on but that's only possible because of open standards API and data. Thank you Audrey and this hour went very quickly and I am sad that it's ending but it was actually a very interesting point Audrey that you ended on because in the throughout the conversation obviously we need both political will and democracy and it's not just about turning on making things digital and so I hope that through this process that we're doing in the Japanese government we everyone is aware that we need to transform the way we think about inclusion in government not just the mechanics and I thank you very much and hopefully as we go through the process I will be pinging you guys separately to maybe help and work on some of the things that we're working on in the Japanese government because I think this is a kind of a once in a lifetime opportunity to make a big change and I wanna make sure that we maximize the positive effects of this focus on digital and not squander it so thanks again for your continued participation and for joining us today. I've got my most treasured work and travel so yeah I'm also looking forward to it too. Yeah so maybe we could hear some time soon. We're all looking forward to traveling finally again. Yeah we will be planning an event sometime this summer so I'll be pinging you separately but I'm coming here. We'll meet face to face next time over Sushi. Okay thank you and bubble tea I'm sure. Okay cheers. Thank you everybody. Bye. Bye.