 Yeah, I think people can do these two together, they will have an idea sharing before this finish. Okay. Then we'll have a public rate and we'll just wrap up in 10 minutes. Okay, cool. So we actually have 45 minutes of like extra time here that we can steal. Okay, great. Okay, perfect, perfect. Excellent. Almost an hour. Yeah. Okay. Yeah. Cool. Awesome. Sweet. Sweet, sweet. I will start wrapping people up. Yeah. It's too cute. Cool. I'll just go to the toilet and I'll come back. Yeah, yeah, yeah. Hello, hello. Hi folks. If you can grab water, grab coffee. Then you can go to the bathroom and return to your rooms. We're getting started. I guess I'm here to color dots. And you know that you start to care for these people. If you want to come down here, there's no problem. Wait a minute. You can use this. This one. Wait a minute. I'll fix it for you. And then you can go up and down. And then this one. This one. This one. This one. This one. This one. This one. There are two ways to line up and then move inside. There are two. Yeah, I'm a little tired. You're really veio simple. Who are you? I just saw this. They're who I was earlier and I was there just now. There's an old guy just arrived, he's not afraid. And he will listen to me. No, you can sit like me. Then I'll go up, hold onto himself. So we will light up and let Xiaoxu on screen. to be able to come out of the workshop in the actually liberation and co-operation session. So in the afternoon, we're actually discussing the actually liberation and co-operation session. We wanted to take pictures. And the exercise that we did in the afternoon in the morning, we were interacting, and they opened that meeting. And those allows us to see what's going on. On the topic, and who are those stakeholders who are involved. So for those who haven't done the statement yet, it's OK, because it's just to give you an idea of how each and everything and how the stakeholders that correspond to each other. It allows you to identify who are missing and who are not included. So because we are running out of time, so we will go into the exercise 2 and exercise 3. I'll give them up. The exercise 2 is about generating challenge statements. So we will do this first, and then do the development part. So as to the liberation process, we will start with reflecting, and over time, we will make mappings. So for those who already post your statements to the drawing closer to it, I think the next stage is the time that you reflect on those statements. If they do between the problems and the idea statements, I think of all of the groups. You have the ideas, but how does this idea respond to any problem? And if it's yes, does this respond to one or multiple problems? So in this session, you have to be critical on those statements. Even those statements that you think it doesn't make sense, then you probably have to rewrite it within the consensus in your group. So take 15 minutes to reflect and reorganize the issues when you did earlier, and then come up with challenge statements based on that. So I'm going to show you about how you can generate challenge statements. Usually, the statements start from how might we. That's why I mentioned that the topic we had this morning is actually a challenge statement. But the crucial thing that we did this morning is to really identify the problems that fit into the challenge statement. Because the challenge statement that we had this morning, it doesn't really respond to any problems, and maybe we think that statement, everyone has their own picture of how the problem statement can look like. But we didn't really get the chance to share those statements among the stakeholders. So that's why the morning session was really important. It shows how people think in a more collective way. So the challenge statement starts with how might we. It doesn't really have a rule that how you can strike the statement. But there is actually some example that you can try thinking how you can structure that. So we can start with how might we and give you a verb, like maybe facilitate or help. And then followed by target audience. So who are those people that you want to help or facilitate, and then to achieve a goal and then under what conditions. So in the statement, it gives you the information that the information that you have to convey is what you want to do. Who are the audiences? And what is your vision? Do you have any conditions in delivering those possible solutions? So this is the statement that aggregates the core problems that you have identified. And then it opens up the ideation that you can do in the following session. So the exercise three, because we're going to go through this all together, so I will explain it at the same time as well. So after you come up with a few challenge statements, in the group you have to think really carefully about what are those statements really resonate with you within your groups. And then use those questions to spark conversation within your group. And then you can start generating ideas. So you still have a few statements, the issue cards on the table at each group. So you can use the issue card to write down your ideas. You can discuss with people, or you want to do it on your own. Then you will have to share within the groups to identify what are the ideas that you want to build up upon. And you have to get a consensus on that. So the co-creation will... We will use the idea development sheet for this session. And the problems over here, you can take the problem statements that you already generated in the morning. But in this stage, only select those ones that resonate with you. And those ones that you think are really important. They are the key issues that you have to address. And put them in the order with the priority. Then after that, you can think about the value. So what values should we hold? Is there any conditions? We have to think carefully before we start generating solutions. And then you can put your challenge statement over here because it's the inspiration for you to think about what the possible solutions can be. And as you also mapped out some of the solutions, these ideas this morning. But those ideas that we mapped out in the morning are generally ideas. And in this stage, you have to be solid enough to be able to communicate how you are going to deliver this solution. And be more specific. And so we can have more available solutions. Also, think about resources. Who are able to provide money? And who are the people that can help you develop this idea? Or is there any existing program? And so on. Take some time to reflect those ideas. Is there any risk? If yes, then put it down. And think about if you have any solution to respond to that risk. And if yes, I think that will be a very good idea that we can keep working on. So after that, you put who can deliver this idea. And each of the information you need to have a link between these. So this problem is actually addressing these problems. This idea is responding to these problems. And this is the resource for this idea. There is a risk, but we have solution to that. And these free people can, or these free organizations can deliver this idea to respond to that problem. And then you can think about how will you major the success? And how could we review the service and policy in the future? So these are the questions that give you more time and space to reflect on, to generate a better policy and service ideas. So we will have, I think, less than two hours to work on this. So I will check with all of the groups after one hour and see how you are. And if you have any questions, please let me know anytime. Is there any question for now? Okay. So I'll hand it over to the facilitators in each group. And yeah, that's great comment. Thank you. One, we have around 10 minutes left. And after 10 minutes, each group will have 10 minutes to present what you've achieved. And so you can send one or two people, decide one or two people in your groups to see who is going to present this. And make sure that you have a link between the problem and idea. So you're actually responding to the problems. Your ideas are actually responding to the problems. So everyone, time's up. And I think it's the sharing session for now. Do any group need more time to finish this? Okay, if not, which groups do want to come first? Which group is ready? Okay, you're ready. Cool. So each group have maximum 10 minutes. Yeah, that's welcome group too. Hello. So our group focused on a cluster of problems around rights and choice and regulations. And then also education around the problems that we identified in those clusters. So if I am not speaking clearly, please help me group. So we thought about problems around if people knew what data was collected about them and how to control that data. So we distilled that into a challenge statement around how we might support NYC residents to achieve control over how data is being collected and used by government and non-government that respects a person's digital privacy and internet human rights. And some of the values that are guiding this challenge statement are language access, inclusion of marginalized communities, access, user-friendliness, meaning universal access, free, easy, quick, responsive to future context and an idea of a Hippocratic oath of data, kind of a do no harm for data. And then connected to that, we're also thinking about what kind of education we need to realize the solutions for the first challenge statement. So we describe that as how might we help NYC residents learn about their data and digital rights and improve ways to protect them. So some of the possible solutions that we talked about were an idea around digital bill of rights, personal data locker, clear designated agencies for dealing with this issue, free city-wide data privacy courses for NYC residents in library, schools, community centers, et cetera, and then an idea around residents getting an annual data collection report that includes information about what government and non-government collected their information and how it was used. So it hits a lot of interesting ideas there around the mechanisms for that. Could it be something that comes to you with your tax return? Something similar to a federal bureau. Could we use other mechanisms there? In some of the other examples, we looked at resources like GDPR, specifically for personal data locker. We're thinking, are there existing things like the ID and NYC that could be used as a repository for that? And then we're thinking about the civic tech community and nonprofits and advocacy groups, think tanks that could help with the education efforts. So a lot of the barriers, there was a couple common themes, jurisdictional issues are a big one for a lot of these. We started thinking about this on the city level, but then a lot of the solutions that we're thinking about are currently regulated on the federal level. So one possible solution is to try to implement a solution on a federal level. Another possible solution is to try to devolve power down to the city level so that a city could regulate something if we think that's a more appropriate place to regulate something. Some of the other barriers that we talked about are who has access to credentials, which is varying degrees for some of the possible solutions we presented. The idea of regulatory capture, if we have an agency that's regulating a private company that has maybe more power than the government agency that's regulating it, then you have the issue of regulatory capture. Costs, some of the solutions to that are taxes, but we also thought about are there ways that we can communicate how doing some of these things would reduce the hidden costs so we can report on the benefits of doing some of these things as a way to help justify the cost of doing them? And then I'm not sure if I'm running out of time or how close we are. Okay, three minutes. I think, yeah, okay, so we're good on time. So the responsible lot is, you know, we definitely thought about it on the local government and the federal government and which way is appropriate for which of these solutions. Involving tech firms, civic tech. I'm trying to think what else is interesting. Yes, I'm not going to go into all of those. But our sort of vision for success is public understanding of their rights around data increase trust and confidence between government and residents. And yes, what, sorry, what? And private public partnership and, yeah, an adoption of the Bill of Rights. Yeah, so any other, any big ideas that I'm missing in here are things that people wanted to highlight. Okay, thanks. Thank you. So let's move on to the next group. Who's who want to go next? So we struggled a bit with the scale of the issue or problem that we wanted to start with. But we decided to use a very specific use case that one of us has encountered and use the tool to think through that. So actually, oh, it's not here. It's this one, right? Yeah. The use case that we started with was being able to find out. So the problem that we had was it's not clear what data sets are for what we're talking about data that the government has and how can citizens use that to learn more about the city and things that are important to them. And so the specific use case that we were thinking about was the data of rent stabilized apartments and that data is out there. But people actually, like I, for example, didn't know that my apartment was rent stabilized and that led me to thinking, well, what are the other rent stabilized buildings in the city? And if someone were to come to New York looking for an apartment, shouldn't there be a really easy way for them to search for an apartment that is rent stabilized? So related to that, the statement was renters need a way to see specific information about apartments like which are rent regulated. There are no clear data sets for that. There is basically a PDF on a government website. So that's not really a data set that a programmer can use and it needs to be scraped and then used. So that's what they currently have. And we extracted other problems related to that such as clean, accurate data set posted. So the need for a clean, accurate data set posted on the state's open data portal. Another problem is just the general usability and discoverability by residents and the verifiability or accuracy of that information. No way to find out if that data is trustworthy or accurate. So the values that we hold are accuracy. We want to make sure that the data can be easily used and not sure what was timely again. That it's published timely, yeah. What's that? Oh, that it's updated regularly. Or can be accessed in a timely manner. So our challenge statement was how might we enable renters to quickly find rent-regulated apartments and their eligibility for them? Or enable renters to find, I don't know, I think we just settled on this one, right? Yeah. What's that? Yeah. So a couple of the possible solutions that we came up with was a task force to liberate or structure the city-relevant data owned by the state. It's basically like someone to play the middleman part to make sure that the data that the city has is structured in a way that is usable for residents. And then a risk and barrier to that is that that is highly politicized. Another possible solution is training tools to help the PDF maker issue data in a usable format. Another possible solution is like a rent bot, which is a chat bot that can tell me whether the apartment, an apartment is rent controlled and whether I am eligible. The risk and barrier to that is privacy. What do we say specifically about, right, just making sure that it's not giving more information than the user wants to divulge? And then also another solution is a map or tool to display the locations. And that would need obviously a tech person or resources to support that. And then the resources that we would need for any of these solutions are, I think this is related, right, for the task force, it's money, obviously, and then a united buy-in or alliance between the city and the state because there are some, they're not completely on the same page. And then for training tools to help the, make the data that's in the PDF into a more usable format would need like, for example, beta NYC's open data portal. Or there's also a website called MI Rent Stabilized, so I think using those things together you would get at that solution. And then to be able to create this rent bot, the chat bot, you would use the, there is actually an existing PDF of listings on the state website. Further solutions for risks and barriers are a therapy between government and mayor, community-driven verification. This is something different, right? And an open, I think a citizen public campaign and focus for open data standards. Some of the responsible bodies that we've identified are the Housing and Preservation Department, Rent Guidelines Board, basically these are all the organizations that have this information, just not in a usable format. Renters' Association to represent renters and residents, housing and advocacy groups, sites like Street Easy, other real estate sites, and New York State Homes and Community Renewal. For the stakeholder map, the core is, on the city side, is the state-led government, and then obviously New York City renters, they're sort of the subject, I guess, of this problem or issue. And then directly, the next circle would be NYC landlords and civic tech devs who would be the ones creating these tools and trying to figure out how to make the data more usable and use that for these tools. And indirectly, private companies like Palantir that collect data, government agencies, philanthropists, and homeless people. Do you want to add something to the homeless people? Those were some of the potential stakeholders that we mapped out as people to take into account during the process, so they're not necessarily contributing directly to the process, but are good and bad and otherwise consequential people who may be affected by these decisions or who could affect the way that the process goes. Yes. So I work for NYC Digital and we are specifically on phase two of a housing portal project focusing on affordable housing in New York and it's a huge problem. I would love to invite anyone from your team and anyone here to join our cross-agency task force. We work with, at this point, seven other agencies including NYCHA and HPD, and we work with the Department of Finance to figure out how to work with the financial empowerment managers when they talk to New York City residents about affordable housing. So all of your solutions, I would love for you to present them at our next meeting and we can tell you what data is available and how we can figure out how to incorporate all of that into our next phase of the housing portal. Anyone here, and if you want my phone number or email, you can give me your cards and I promise you I'll email you tomorrow or I will give you all of my information right after this. Thanks. That's great. Thanks for the second group and now is the time for the third group. Who is presenting? Just following up there, I don't know if you mentioned it, but there's a website called mirentsstabilized.com and they did that to file a FOIL request and kind of walks through your process of checking whether your building has been stabilized. I'll present the first half of our process. As you can see, there's a little bit of a gap posted here, mostly because we actually turned into a really interesting discussion about the meta structure of the process itself. So we'll kind of have a two-part presentation. Really quickly, the core problems we talked about were opaque or bureaucratic processes for data. I don't know what data is accessible or aggregated by myself or by others. A high percentage of population is unreachable or it's unclear how I benefit from the government having better access to my data. So essentially the two challenge statements we consolidated from these core problems were how might we help residents understand the data that the city possesses and how it's used and how it's biased and how might we help community to access use or create the data for public good. And the possible solutions we came up were mostly trying to okay, one of them was a consistent metadata standard and a notification protocol that alerts and notifies people of eligible programs or services. So kind of tying into the first one what we thought it would be like hi, your apartment is rent-controlled looks like you might not know or hi, you're eligible for free health care you're in a demographic that may be eligible for free health care just to let you know or some way to better connect people with services. We talked about using open source CCAN, community knowledge something network software for like a data repo and we talked about this earlier and this earlier phase of our process was around having very easy to navigate accurate websites for the public because most of our solutions hinged around not so much like having open data portals but the gap between knowing what's out there and how what is a good and helpful user experience. So like sites like open data and software development team to do neighborhood outreach. So that's kind of what we were going through and in the process actually turned into a really interesting discussion around the general structure of the development sheet and what it optimizes for so we can talk about that. So I think we started talking about the design of this model itself because we found ourselves experiencing this kind of struggle going even through the first phase and I think it all became more clearly once we were asked to now that you have all these issue statements how do you prioritize them and the first thought was okay why don't we just say what's more important to us but then we kind of took a step back and say no wait what's this process for this is to then open to the public for more ideas and for their feedback so which of these issue statements is actually something that we already know the answer and we just need to get better at it and which ones we don't know the answer or the answer that we know needs the other feedback loop of if this is a good idea are we actually addressing the problems that are harming you etc so for example when we said like help presidents understand what data this the city has about them it's like do we know what data is collected by every agency every department every service how that service is using it we don't probably I don't know because we're not them but we thought like we so that's the first problem and that's a problem in a way we need to solve internally first and how are we going to do that it's going to be a completely different process of ideation that then imagine that we did have that data and we we actually know then how do we open it to the community and also raise some questions like who's over represented by this data who's on the represented by this data how are we using this data to solve problems more effectively so that's how kind of the feedback that we started to struggle with that brought us kind of back and forth on rethinking the model itself and saying how can we optimize this experience it is our understanding that this is for public officials first so what if instead of asking public officials to already start thinking about possible solutions we can reframe every area into problem definition what are the solutions you're trying so far what are the resources that you have so far available for this problem like what's the open data related to the problem what are the constraints in terms of budget in terms of jurisdiction in terms of law etc and in that way when you get these to the community you are not guiding them already to the possible solutions that you already drafted for them and then give them more open and more space for creativity for the community to suggest different solutions instead of guiding them and that's like when we start to okay that would optimize for new ideas problem solving but this model can also be very useful what if it's about optimizing for consensus and then actually the public officials are able to say this is something that we don't have an answer these are the possible solutions out there we are not arriving to consensus because of these risk and barriers let's open it to the community to see if there's more possible solutions and then what we need to do and what we need to optimize for is consensus and how would we do that so it was just more of a conversation about the tweaks of the model how can it be used differently depending on the ultimate goal that you want to get to and that's the result of this team's work yay well thank you for sharing I think we will have a coffee break for 20 minutes and when we come back I will just briefly talk about what will happen after this and just 10 minutes and then we will have around we will have 45 minutes to talk about what you've talked about how does this model can be optimized what do you feel about the experience that you have today and also our PO's will share about their experience as well so we will have a more open discussion your feedback and your feelings and any suggestions that may occur to you today so yeah so let's take the 20 minutes coffee break on small announcement we have tried to make everything vegan veggie friendly but I do want to warn that the rice crispy treats have pork gelatin in them so if you are really into that then you should grab them so there are snacks out there coffee and water back to work Noel Hidalgo time to get to work Noel why are you standing here Noel okay just an interruption I work for the internet society which is kind of responsible for the internet and the history of the internet was a department of defense experiment which grew and then became a research thing that was run by the national science foundation until about 92 when it became commercial and the national science foundation wanted to let go they let go the backbone over to MCI and so on and basically the whole thing was run by a mailing list called the IA internet engineering task force the IATF and the internet society was formed as basically organizational backing for the IATF so that they could have meetings and at the same time in 92 they actually the IATF this guy Dave Clark wrote a code which you can see here an aphorism we reject king's presidents and voting we believe in rough consensus and running code and that is the the motto of the IATF and that was in 92 this one you know so the system is they do requests for comments and then people comment on them and this is RFC 7282 from 2014 so how they make consensus in the IATF is that they hum so they don't vote but they hum and everybody the more people hum the more you know they know whether something's good and if people groan or something you know so that's basically the way that it works so in the 2000s this was known as the internet model multi-stakeholderism and sort of more recently that we've come to appreciate but this isn't really the internet model it is multi-stakeholderism and a prime example of this where it really came into the thing was the actual control of the of the root of the internet the what's known as Diana function and which was you know is controlled by the American government and so after Snowden America was trying to make good with the world and so they said okay we're going to hand the root over to the internet community if you have a multi-stakeholder process involving everybody and no government gets to control the root of the internet of the main name system so they had this thing and eventually it did happen Ted Cruz tried to stop it and didn't manage and even just two weeks ago they issued a notice of information where they're trying to see if they can take it back they can't and the guy who handled this was a guy called Larry Strickling who was the undersecretary state for the department of commerce and he saw it all through and so he got fired when Trump came in so he brought him to work for the internet society and he's now running something called the collaborative governance project for the internet society and basically he goes out and preaches multi-stakeholderism he's looking for techniques I write to him and say look at this Taiwan thing I have written to him more than once and he's beginning to think about it but he made a speech last week which was just a call to action for multi-stakeholderism which is this short video if I can find my mouse anybody can see my mouse on there oh this is the one there it is I can get to there and switch on audio here and find the go button deliver a call of action to each and every one of you a call of action that each of you commit to get personally involved in a multi-stakeholder effort to address key internet policy issues to find consensus approaches to dealing with the issues and to implement those solutions now is that too much to ask at 9 o'clock this morning I don't think it is but to do this you're going to need some help and I'm going to provide some help right now to those of you perplexed and confused by the multi-stakeholder process first we need a shared understanding of what is the multi-stakeholder process and we need a shared understanding of what it is not now there is no single model of the multi-stakeholder process but for purposes of this discussion let's focus on the key attributes that define an authentic multi-stakeholder process one it must be stakeholder driven in that stakeholders determine the process and scope and direction of work two it must be open and inclusive both in terms of allowing broad participation and in ensuring that all issues are addressed three the process must be transparent and accountable for all stakeholders and to the public and four outcomes must be consensus based delivering positive value to the greatest number of stakeholders now a multi-stakeholder process that exhibits all of these attributes is most likely to be one that will be accepted as legitimate by the stakeholders and the public just as important as what is a multi-stakeholder process is understanding what it is not and one it is not an ISM the multi-stakeholder process is simply a set of tools to help people collaborate to solve problems it is not a philosophy or a political ideology and we do the process of this service to refer to it as an ISM because it encourages people to argue about it and take sides about what should be a set of tools that can provide and help everyone secondly beware of those who haphazardly or even manipatively attach the label multi-stakeholder to what in fact or multilateral are top down processes when a government or a business runs a consultation that is open to input from all stakeholders but keeps the decision making to itself it in no way is running a true multi-stakeholder process and we find that rarely do such efforts even allow for collaboration among the stakeholders much less decision making by them so having described what is a multi-stakeholder process and what it is not let me turn to my second point which is that people are reluctant or afraid to take the initiative to establish multi-stakeholder processes part of that fear stems from people not knowing how to organize and manage a multi-stakeholder process and that can be easily overcome the internet societies collaborative governance initiative is developing training materials on how to run multi-stakeholder processes and there are already lots of instructional materials available we in the internet community may like to think that the multi-stakeholder process was born with the internet but that is simply not the case collaboration and consensus building techniques have been around for decades and there are many instructional guides and manuals available worldwide to anyone who needs some help but trust me when I tell you that of the keys to successful multi-stakeholder process they mostly reflect simple common sense and basic stakeholder management principles it's not rocket science and it is easy to learn another part of the fear and reluctance to bring forward multi-stakeholder processes stems from the sense that the problems today are so large that we protecting privacy, battling cybersecurity threats and the like these are so large that people are intimidated about trying to organize a response also there's a predisposition to want comprehensive enforceable solutions to these problems which usually translates into waiting for government or some other organization to take the initiative I urge you not to let that fear deter you from taking action the existing government processes to enact comprehensive legislation or write regulations or negotiate treaties are not well equipped to deal with the fast changing issues of the internet these processes are slow they usually don't result in any outcome and even when they do the problem that they intended to address exists and if they get it wrong it's incredibly difficult to undo bad legislation or regulation instead I encourage each of you to think about how to address internet policy challenges in smaller bits can you define a piece of the issue in a way that a group of stakeholders in your own country or region could address the topic find a consensus solution and then have stakeholders implement it we need to encourage experimentation around the world we need to encourage finding ways to make incremental progress on issues by solving parts of the problem and then building on the small successes through iterations of the process over time you will need to be creative you will have to focus on elements of these issues whose solutions are within the power of the participating stakeholders to implement let me say that implementation is key too often participants and multi-stakeholder processes claim success when they reach consensus on the solution but then they fail to address who's going to implement the solution and how it will be implemented you must focus right from the start to define the problem to be solved in a way that allows the parties to the process to implement any consensus outcome as an example what I mean, take the issue of disinformation or fake news online some of the responses being discussed involve creating comprehensive regulatory frameworks for deciding what is disinformation and what to do about it absent a government convening the multi-stakeholder process to create such a framework there is little use to having you organize your own process to develop such a scheme or ability to implement the framework and because of that fact you would have a hard time sustaining such a discussion for any length of time but that doesn't mean it's an issue you should stay away from you can make a contribution to solving the problem what if you convened a multi-stakeholder process to develop and implement an educational program on media literacy you could certainly identify a set of stakeholders who would have the ability and incentive to design and deliver such a program now none of you can do this by yourself you need to work together with others in your community you need to recruit stakeholders who will represent all of the key views that have to be addressed but it takes a catalyst someone who steps forward and takes that lead to initiate the discussion and each and every one of you can play that role as a catalyst no matter how narrow the issue or how limited the geographical extent of the discussion every successful multi-stakeholder decision that is implemented provides input into new norms for answering the challenges before us over time they will combine and compound into broad solutions and if and when governments decide to legislate a comprehensive and forcible approach they will be guided by the aggregation of these individual solutions commentators refer to the internet as having allowed permissionless innovation the idea that allowing experimentation with new technologies without requiring government or competitor approval what I'm urging all of you today is to engage in permissionless problem solving if we all make it our responsibility perhaps we can spur the next great leap forward for the internet to better recognize and respect human rights to better deter cyber crime and cyber attacks and to better meet the needs of citizens around the world and all I ask is that you give it a try so please do it and I thank you for listening to me this morning I come before you top of the video it's on top of the video now it's in the url now it's in the green now it's back to the video now you're dragging the window you got it? thank you for sharing this video it resonated with our exercise a lot from today and I'm going just to wrap up what we did today and how does this align with our collaboration workshops so there's something that we didn't demonstrate today was actually the preparation how can we turn the preparation into the material that we used during the deliberation we didn't actually stimulate the part how people deliberate on those materials that we prepared because it requires like two or three weeks time to get a chance to think the size of the information reflect and critique on that then present it back to the wider stakeholders and then they can add extra comment on that so and also lots of people might be confused about the first exercise so I will explain a little bit more about why we're doing that exercise so the exercise one is actually the training part of the whole process like imparting the severe servants to be able to be equipped to tackle diverse statements from any kind of source so that's the training session for them and then after the training session outcome will be aggregated being synthesized on this big canvas so this is actually the working document for the severe servants they need to put all of the statements in a structured way to make sure that these includes all of the problem descriptions uncertainties existing solutions so they will map how all of the statements over here and also provide the evidence so we make sure that if we have any question to those statements we have someone to refer to and then the second part of this document is identified stakeholders which you all did this morning and this allows us to understand who make that statement and also it allows us to understand and think about who are missing during the whole process and the third part is the critical part because as the first group mentioned that there are some existing the existing solutions from the government that we should understand beforehand so this is actually the part that we ask the severe servants to fill in what are their current plans that respond to all of the problems on the left hand side so we get an idea of what's out there and then we also push them to think what they can do more because when we ask them to provide what they've already been doing we also ask them about their difficulties if they already did a good job then people would not raise the petition or the problem so there must be something wrong with what they're already doing and we need to know why it's not working so we also asked their point of view of what they think is not working and if it's not working then what would be the best way to make it better so we also do the during the preparation part within the government across different ministries we also ask them to think future plans and what other possible resources they can use so when we all have those information before the actual deliberation during the co-creation workshop everyone can get a good idea of what's going on and what we can build upon that so this is something that we didn't do today and if we actually have this big plan before the deliberation I think things will go more smoothly during the exercise one and so after that I can show you the real-time words over here so in the actual preparation stage we will finish this big canvas the whole picture research and then turn that into a mind mapping a mind mapping diagram that we show this in the beginning of the deliberation so when people come and see those information they can actually add their comments on it so we don't actually give people post it people can write on post it but they can also talk about it and we have our colleagues writing the comments on the digital version of the post-it real-time when somebody talk about something so everything is being documented real-time and being synthesized at the same time so people don't just talk about things come from nowhere, they actually have structured information that can discuss by different order and they can also this prevent them to talk about same thing again and again and this will force the conversation to be a more solid situation so this is the part that we didn't just demonstrate and let you practice either and so during the deliberation and co-creation part we will actually present this from people the civil servants from different ministries will do presentation based on the structure so people can understand the mind-mapping better because not everyone is familiar with this but they can listen to the slides from the representatives from the ministry so usually the deliberation starts from 10 o'clock and finishes at 3 o'clock so 10 o'clock we will welcome everybody to talk about the process of the workshop then the civil servants will talk about the slides based on the mind-mapping structure then we open discussion from all of the people so we will have everyone looking at the mind-mapping board and starting talking about the problems that is not discussed on this map then we will have a very good view of what everyone thinking about until around 11.30 then we will start think the sizing of the information and generate challenge questions like what you did today so during the lunchtime we will have a few how my way questions and then in the afternoon from 1.30 to 3 o'clock we will have time to do the co-creation like what you did this afternoon so what happened after that these are all initial ideas and possible solutions but it's also important to reflect again again does this really work so after the co-creation session our colleagues who also participate the workshop will generate this document for policy makers to understand what people were discussing about and yeah the question just about the categories of the comments it took our group a little while to figure out what categories to use and I noticed that we're similar but different to the other groups and so at one of these meetings is that too late to re-categorize issues or are things moving around and new categories being added and removed so during the preparation stage within the government there's a lot of debate on how to categorize those things so we gradually build a consensus of what the category can be among different stakeholders within the government usually when it comes to the revision part it's already being polished and people generally agree on that but sometimes they will add some more categories that the citizens haven't think about so it's really good to have the deliberation to have the deliberation that allows people to add more comments on that but usually we don't we rarely change the category because it's being polished again again and be deliberated within the government for a few times so speaking of the policy canvas it allows the policy makers to make sure that the process during the co-creation workshop is actually being able to connect to the actual policy making process but this is not still not enough when we want to reach the deliver stage like what I mentioned earlier about the double diamond process the policy canvas is probably something around here and we need to make sure that we have a good understanding of how we can deliver that better because there are those points that what can be done in the future but how can we do that and how do we measure the impact and how can we make sure that people can really benefit from all of the solutions or can these really respond to people's problem we need to carefully think about that during this process so it will usually involve lots of meetings and editing different words and also include the comments from the frontline staff so they usually give a really good view of how these things can work so and when everything is being identified and being made sure I think that's the point where we can deliver that but the process doesn't usually end over there we expect people to be able to keep irritating the process even when the project is delivered but it's really difficult within the within the government and so yeah our wrap up over here and now is the time for open discussion so if you have any thoughts and any feedback just feel free to let us know because we actually the idea development sheet we iterate the model every month and this has just been finished a few days ago so we keep updating our process based on the experience from the participants interview any kind of stakeholders after the workshop and understand what we can do better and what we are good at doing so it keeps the model growing and we want to optimize this model just keep iterating and try new things and we feel we learn so we want to keep improving so today we also would like your feedback as well and so during this session me and Patricia and Shuyang and Tiffany and Audrey will take questions from everyone so who want to go next hi this is very loud my question is about prototyping and how you present some of your policy or design ideas beyond text or the next level of fidelity so for example on these sheets you have a space for like pictures and images and you use this design double diamond process here so I'm just wondering are there ways where you sort of present design prototypes other for tools or like policy drafts for people for comment like what happens after this process so in regards to the online text system that I present earlier it's vary from projects because that one is mainly about how we can improve the public service not necessarily responding to the policy change so in regards to that project we develop the prototype based on how we create a digital service so Shuyang was also part of the facilitator and coordinator of developing the tools that allows different participants from different background being able to do the prototype so we design the prototype tools specifically for that project and depends on the project any questions sorry I can't read anymore but about the rapid testing do you actually like how really is this testing do you create pilots and then create proof of concept and then replicate and scale or is it just in paper during one session so there is only one I think how many projects have we done pilot the text on my text system so it's easier with digital services like the text file system and the Medicare system the CDC system there's like three or so that we did pilots but if it's about urban development or rural development or whatever we only did mock up or simulations it's not a real like a full ethnographic project we haven't done that yet I think that's a very critical thing because there is no such imagination we think the government yet about prototyping a policy like and that's something that I would like to work more in the future to be honest because I think there's still no room for this so for example the policy canvas that I showed earlier from from the Ponghu project and I didn't actually involve the policymaking process after that because we do have we have to take all topics different topics every month and we at least have two projects to work on so usually when I finish the aviation stage and have seen how it's been translated into the policy canvas then I will move on to another project so I also would like to know more about how actually civil servants take this further so this question can refer to Patricia because I'm also curious to know how they actually act on this canvas after so how does the council of agriculture take this policy canvas forward in the in the real practice how does the council of agriculture take this further like after we do after we've done the cooperation project believe the colleague Yutang has been transferred all of the outcomes into a format that is is more more friendly to the policy makers and so how do the policy making process being done after this translation I remember that after this collaborating meeting they left some homework they many ministry take home and the part of our fishery agencies is to do some research on this issue so now in lawmakers we have no no any further but we keep concerned on this issue and the main homeworks left to the ministry of interior the marine the marine national park headquarters they have to communication communicate with the fishery community and the diving lovers I just remember that so because the main issues that we were discussing about sorry I didn't touch too much about the content of what we actually discussed on that project but the main issue that has been raised during the workshop was should we limit the amount of the fishery industry and also like the tools that they use to capture those fishes do we need to make it even more harder for them so but in order to make that decision we need to research a little bit more on the environment perspective and there is a limitation of the documentation and research around that should we really protect that area the sea area over there nobody can really answer that question that's why their next action is to do to outsource the research to the company so that is not something that can be prototyped but they can actually have an action after that and that is one of the problem the solutions that we came out during the workshop that has been turned into an action but in regards to some of the other policies I think there is still room for them to do some tests so for example what if they really restrict the behavior that if they really restrict the fishery industry then how can those people survive because they might lose their job or they will not have enough income so probably we need to find something that compensates that and how can we do that we actually have to prototype that but I still didn't see that happen but I think that's something that we can work on a little bit more before it's been launched like the government can say oh this is a great idea and we should do that but what if fishermen doesn't want that what if fishermen doesn't want to compensate in that way I think that's something that we can prototype on and we have to work more on that thank you so for when your projects become a policy who or which agency or department is writing that policy is it a co-design or co-writing process with the public and before it becomes an official policy does the public get to review it one more time to give their comments is there any iterative process with that so during the co-creation process it's actually the opportunity that allow there's an opportunity to that different stakeholders to write the policy together and after that there's the people from the from the civil service they will translate the outcomes into the policy terms and they get the chance to review if it's going to be implemented take further or not but that's also the critical part because sometimes the participant not all of the participant get the chance to involve in that decision making process but sometimes they do invite them to come to the meeting so it's varied from different ministries so given that we have some participation officers here I'm really really curious this is a little bit more of a intangible question but not so much the specific policy changes but how have their own perceptions of public participation and of the public shifted say in the last you know six years since you have been doing all of this would somebody be open to speaking to that? I think Patricia actually have a slide to talk about her experience working as a PO and what she learned and what what the challenges are so I think she can touch on that question later on one idea that came up in our group was just a difficulty I think I already said it but putting things into categories and when you mentioned feedback I was interested in what a tagging or we started to think about what a tagging system might look like because so many issues seem to cross categories and especially if you're working in a digital space is there some other way to spatialize issues so that they're not so strictly bucketed so I thought maybe interesting we do sometimes face the period that we cannot really categorize some of the statements and usually they are critical like uncertainties so we will categorize them in another category and say that they are un-categorized but usually we want to categorize them as as much as possible because it gives us a sense because there are so many when we are facing a really critical topic we usually get lots of problem statements and the category allows us to discuss the issue in particular order so people don't talk about things like oh I talk about privacy and then the next person will talk about how can we make data like how can we best utilize data and then the conversation will go nowhere so that's why we want to categorize things but in regards to how we do that it usually it's there's no certain methodologies to do that it's just lots of time of selecting, criticizing and really trying to make sense of how we organize those data so usually just take time to really understand the whole picture and then finally we can have those categories because sometimes problems if they say there are three people want to address one particular problem people can address that in different ways that's also part of the process to try to understand what people are really talking about and then by doing that we can reduce the statements because we understand the situation and we try to use the statement that can contain most of the thoughts in terms of the model itself and in pilots and testing the impact of the model like I understand that there's a common structure or architecture and then based on the needs and the problem itself you make a lot of decision choices or design choices out of it like oh for these we do a pilot for these we go to the communities for these are you documenting making these design choices and are you in a way measuring impact as to this has been more successful that's a really good question both things are the things that I'm currently doing so I'm writing a paper about the things that we've been doing for the past two years and try to document the process and also in the paper I will be really critical about what we're doing and what things can be improved because there's still a gap between the ideal process and what we can do due to the restrictions within the government so yeah I'm happy to share with you the paper once I've finished and the second question is about measuring impact there's maybe testing or so how we do the testing is actually today is also a part of our testing as well and we use the following workshops to test our approaches so we have 36 collaboration workshops so far and every workshop is slightly different because we learned something from the previous workshop and we test something at the next workshop and we do document those things using the mind mapping it's a mess and we're still I think all the information that we aggregated we did lots of interviews so we keep all of the transcripts in our document and we are still at the stage of analyzing people's feedback and if there's something that is actionable we put it into the next workshop and we test it and then we learn from that can you talk a little bit more about how you debrief with the team after one of these sessions do you have structured notes do you have a structured process for how you do a debrief like what do you do it for two hours the next day so how do you factor in the feedback you get how do you debrief with the core team that does that facilitation within the core team we do have a document that have a list of what we have to do before the workshop and during the workshop so we have a document for internal colleagues to understand the process but it's not it's written in Mandarin so I cannot show you and then after the process do you ask a certain set of questions that you evaluate yourself that you evaluate the process by do you take those notes do you have some industries we call it like a hot wash where you try to get people to express their frustrations with the process from within the core team yeah so sometimes we have internal meetings after the co-creation workshop just to get everyone core team's idea about what they have been experienced and what can be improved and we document all of the discussions does that answer your question yeah just have all these nice templates I was wondering if you have like a worksheet for that oh yeah we do there are two ways of documenting for one way is Google doc that we write down what we've been discussed and we put that into format so actually there are a few discussions that we have are they're so informative so we actually put it in this way like we use issue mapping instruction to put in like we use this structure to identify and document the problems that we can improve and so yeah this is the way that we are working on and also another way is to turn this into a mind mapping diagram so we can see those problems and what we can improve in a more holistic way so mind mapping and issue mapping instruction of the 36 workshops that you've held so far how many are there with the same participants or the same type of participants the same places and or the same issues in different places like I'm just curious how much are they a build on versus a completely new process from scratch so to say in regards to the participants usually they they are vary from different workshops because the topics are different none of the same so if I participated in your third workshop I'm not participating in the 10 usually not just one okay yeah and even even that the ministries that are in charge of that particular issue because there are 31 ministries and some of the projects they may attend workshops for a few times and even though there are only a few severe servants from that ministries have been participate in the workshop for more than two times I think usually so far as far as I remembered apart from PO because if they the issues is related to their ministry they usually have to come to the workshops so usually the severe servants who involve in the workshop are the PO's and then there are severe servants who are particularly working on that policy or service they will come to the workshop but even they are not usually coming here because the government is such a big organization not and everyone is working on very specific things yeah have you done any research with citizens who participate in these processes understanding afterwards does it change their attitudes towards government or the democratic processes or like how that kind of citizen level cultural change is happening so we did interview a few projects that we actually have solid outcomes like either policy change, low change or service change and those participants they find the process really informative and lots of them give us feedback on the issue mapping process that they find it interesting and also it allows them to have the opportunity to think and also reflect on different people's view so they actually learn something from that process and they get the chance to co-create possible services and solutions but we didn't really speak to those people who are a little bit marginalized during the workshop like there are also people who are a little bit shy and they don't talk a lot and maybe they didn't participate enough they just watching and observing we didn't really get the chance to get those people's feedback and I think that's something that we can work on a little bit more as well maybe Audrey can also feedback on this project just a quick question extension do you at the introduction of these types of workshops do you talk about any code of conduct any very explicit like we want you to participate we want you to be neutral in your types of tone yes I do and I also explain the process and the tools that we use and why we're using those tools in the beginning of the workshop where can I get a copy of those is that in the slides um yeah I can share the slides on to the hack folder later on and any difficulties that you find during the process is there anything that you particularly find confusing or any at any point where you feel like you need some support and clarification is there anyone want to feedback anything on that yeah so for me I think the stakeholder map is a little confusing with the core direct and indirect and just in our group of doing it we when we wrote out all of the different stakeholders I don't think we really understood what direct indirect and core meant at the beginning but we figured that out later on just I thought that was just slightly confusing and how did you figure out I think we decided to make our own definition of what that meant actually it differs so it really depends on the group consensus some people would like to put those people who are going to deliver the ideas in the core so they are the core member of delivering the solutions and some people do like to put the people who are going to be affected in the core as well so it depends on who you want to put in the core and then the stakeholders around it to build the direct and indirect for me the aspect that can where I see more area of opportunities on the problem in my experience working with cities people assume a lot of problems but they don't test if they are actual problems or not so I like to take more like a hypothesis driven approach in which we do help and facilitate the brainstorming of problems and then help with the distinction between symptoms, root causes solutions disguised as problems themselves and then ask them to go back and look on the data and test if that's a real problem or not and actually like share a story because sometimes we can get so lost into problems and it's like ok who's affected by this problem tell me their name, tell me their age tell me what implications do you think this problem has on their side and then we open I guess that you do it all all together but we do it only on the problem stage like go talk to the people affected by the problem and make sure your assumptions are real kind of assumption checking and more hypothesis driven problem solving it's I don't know if this is something that you want the second stage and not on the research stage but I would suggest during the research stage so during the preparation stage, sorry I should frame the exercise one as the guidance stage so the actual preparation stage will be lots of interview lots of interview with different stakeholders like experts the people who will be affected by the policy or services and really get their view beforehand before we have the workshop so we do some clarification before and it allows us to be more aware of what people will talk about during the workshop and also help us to identify the facts really quickly and lots of literature review as well to make sure that we have all of the facts as many facts that we can have I think that the process was kind of amazing because at least for our group there was just kind of total chaos for a period of time and then eventually things kind of snapped into focus and I think that was a really big learning moment for me that sometimes you need to have patience with that process and that it's the role of the facilitator to kind of hold space for that ambiguity until things start making sense but at the same time I wonder if there could be a slightly more structured onboarding process because I think at the beginning you have this real tentativeness when the stakeholders don't necessarily know each other, don't know what everybody else's kind of take is in the process and what you tend to end up with in those situations at least here is that people who are naturally loud and talkative like me will take over and then people who are more trying to listen and make sense of things don't necessarily get a chance to express their thoughts so I wonder if that could be more structured a little bit to frame that part of the messy process that I guess yeah that's a really good suggestion thank you so much to add to that question you mentioned people who are shy don't necessarily come at as much so like what do you especially in places well honestly like the United States where the loud people take over and the shy people just back away what do you do to draw out the sentiments of the people who naturally just back off so it's really important for the facilitators to do some research about who is coming before the cooperation workshop so they get a sense of who will be talking about what it doesn't we don't often we don't usually get information from the citizens particularly because some of them may not want to tell us about what they are doing their perspective it depends on people but in regards to the project that I mentioned earlier when we go to the Pungkula Island on the southwest of Taiwan we reach those fishermen and the fishermen's associations so that's another way that we get a chance of what they might be talking about during the workshop and it gives us more awareness so it depends on if we really want to facilitate the discussion I think a certain of preparation about the background of the participant is very is very crucial but we don't often we don't usually get a chance to do that so we set up a Slido channel for people to input anonymously or pseudonymously just while other people are hogging a microphone then can just write arbitrary things and if it's a live stream it also appeared a live stream so in that regard it's exactly like V-Taiwan and around noon time we also dedicate a few maybe 10 minutes or what for the main facility it could be me or Billy or Feng to look through all the pseudonymous or anonymous Slido comments and try to resolve it in the mind map but the problem is that because it's anonymous or pseudonymous it's impossible to have follow-up interviews with those shy people because we don't even know who they are but on the other hand the fact that we don't know who they are is safe for them because they don't have to review their identity to propose such comments it's pretty simple what's your internal training within your team you're testing out an exercise or perhaps like training other people within your team what's your process for how you kind of share your internal skill sets because what this is is kind of like a simulation of the practices that you do already but within your team what kind of exercises do you do within the exercises we also include training and simulation just like today yeah so it's really much similar to what we've done today okay so I think it's time to hand over the microphones to our two lovely POs and I would like to firstly invite Tiffany to share about the our e-petition website join and if you have any questions about PO's experience and and anything related to join you can also refer the question to Tiffany and Patricia hi everyone I'm Tiffany I work for the national development council and I work in the department of information management I know some of you are very curious about the e-participation mechanism so this afternoon I will give you a very brief introduction about this mechanism and if you have any questions I think Audrey will answer all questions because she is my boss okay but it doesn't go to the next page it doesn't work sorry wait a moment page down page down it doesn't work how can we go to the next page maybe we can change it sorry wait a moment next page okay so how can I get the story of the e-participation let me go back to the 2014 at that time the NDC hosted the economic and traditional conference people suggested government should set up website regarding to public policy issues but reference to the White House website with people I think with the people you are very familiar this perform and people would like to request government to establish the national media proposal center so after the national development council launched the online participation in public policy we called it join.gov.tw so the goal of the online participation in public policy we have three goals we would like to achieve first is to feedback from the city, second is concatenated network entity and community multi channels and the third one is to how can we to complete the process of environmental response okay this is our concept of online participation in public policies like a virtual circle of societies we would like to listen to public opinions and encourage people participation and in terms of online interaction on the social media we would like to integrate the most popular social media social networks to enhance interaction with people so in terms of user friendly interface and the compatibility of different platforms we would like to try to design a flexible way of design and then people can use any device any mobile device to access this platform and we try to use the infographic and because it's easy to read and easy to access so that's our core concept of online participation in public policy so on this slide you can see the online participation in public policy we offer our citizens for kind of services which are proposals consultation department directors mailbox and monitor projects because today my time is limited so I just want to focus on top two which are proposals and consultation and as you know we think the platform is just kind of tools we need to make regulations and so that all agencies can follow our requirement and can follow our concept to implement those things so we met two directions one is directions for implementing on participation in public policies and so you can see the proposals proposals include five procedures for proposal identification proposal submission review registration of support and proposal response and in terms of consultation on policy and driver laws and regulations we ask agencies when planning major policies or raising issues of social concern the responsible authority may actively use the participation platform to consult the general public and facilitate the focus discussion by the general public so in terms of administration as you concerned about the CIO role in this mechanisms so we ask deputy head spokesperson or secretary general of responsible authority is responsible for supervising matters related to these directions and a responsible authority may form a working group for undertaking the requirements of the public policy online participation mechanism and finally the participation platform administrative authority should regularly compile a report of all proposals and the status of policy consultation responses for the representation to the executive Yuan okay every every information on the platform should be open on this website okay and the other directions is about implementing the role of the participation officers in the executive Yuan and subordinate agencies I will skip this slide because I think Patricia will introduce this mechanism so I will skip this slide okay this is our full chart for making proposals and registering support you can see this full chart there are in total of five steps to make proposals first is about proposals identification and the second step is proposals submission and the third step is review by the NBC and the fourth step is about registration of support period and finally is about authorities response so you can see these are our steps okay I will elaborate all steps on this slide first step proposal in terms of proposal identification a proposer may submit a proposal online only after the participation platform has conducted at one time authentication of the phone number and email address provided by the proposals okay and the second step is proposal submission we do not deal everything because we are executive union so the topics are limited to affairs and business related to the executive union and its subordinates so we do not deal with like a political issues or national defense issues because that's a president's right so we don't touch those business and the second is the initiator can choose up to three authorities which should be responsible to the petitions okay and the review stage the participation platform administrative authority that's NBC should conduct the review in accordance with the substance of a proposal when necessary may invite the responsible authorities to assist the reviews and as a rule should carry out the review procedure within three working days so we will have a rush time to do the review and the step four is about the registration of support or persons processing the citizenship of the state or holding a certificate of a permanent residency in the state may use a variety of accounts for logging on to the participation platform and after one time authentication by means of mobile phone number register support for a proposal to become established as case requiring response a proposal must receive 5000 registrations of support within 16 days okay and let's go to the step five it's about the responses the responsible authorities of each proposal should pay to opinions from all quarters and access the visible of operating the proposal into policy implementation the responsible authorities should have a time period of two months for processing and responding to an established proposals and finally the formal response in the explanation may be issued through a press conference or other means of information the general public information on the response should also be openly posted on the participation platform okay next this slide I would like to elaborate how the responsible authorities will deal with the established case and you can see a special thing when the day 14 a responsible agency will contact the proposals to clarifying the peers we would like to understand his needs so we would like to see the proposal in person and to understand his needs okay so after 60 days the government will produce a formal response and I think there are four possible consequences which are taken into consideration it may be partial accepted or we will totally accept the idea and sometimes we will dismiss these ideas so there are four possible consequences on the response I will show you some data about case scenario of proposals till now there were about 5700 proposals by the end of May 2018 and 2016-22 were enter support registration procedures and 144 were established cases the established rate was 5.5% so and you can see the figures it's about 51 cases have been taken into account 31 cases have been partially accepted and 48 cases have been dismissed and there are 11 cases under investigation so people are most interested in these areas like education supports and transportation and infrastructures health and social security these are issues people more interested in now I would like to introduce some established cases it's very famous in Taiwan first is the act to amend the law of immunoscied therapy to the legislative by the end of December 2015 this is our first case and it's kind of accepted case because the ministry of welfare and health make a law about the they allow the cancer immunoscied therapy into the law so I think it's a very famous case in Taiwan and the other case you can see prohibition on the use of disposal because we would like to make our country more sustainable so there are big issues in Taiwan so I don't want to explain every case but you can see the proposals workshop as you know some proposals are very complicated in the very close ministries so my boss coordinate workshop collaboration and make cooperation with other ministries to produce a good response to our citizens so you can see the pictures of the workshop okay this is my I don't want to elaborate okay and then I will introduce the consultation consultation on policy and drug laws and regulations we provide three kind of service on consultation which are policy consultation drug law consultation and drug regulation consultation okay and the establishment of government and citizen network participation in the dialogue mechanism during the policy formulation process providing a network channel for consulting various opinions before the formation of department policy and in terms of drug regulation consultation since 2017 the public notice has been published in the public policy network participation platform to gather opinions from the public on amendments to the draft orders this is the figures you can see on the slide so on this slide you can see hot topics for discussion including like comprehensive renewal of the multi-card unified national identity card in Taiwan we are doing to replace our new ID but in the past our ID is printed ID we would like to transfer the ID to electronic ID so some people think can we merge electronic ID with our e-health car so there are a lot of discussion on this platform and in Taiwan we are also concerned about the homosexual people a manager like or manager relationship in order to protect their rights and the legal status so those issues are very hot in Taiwan so in order to understand the satisfaction of online participation in public policies NDC conducted a survey from September to October of 2017 you can see there are 82.4 of respondents are very satisfied with this platform and 82.2% of respondents are willing to recommend this platform to others and there are 43.5 respondents think the platform has a certain impact on the policy but 17.2% of respondents think a little bit improved trust in the government so we will do the certification surveys last year so that's my brief introduction about our mechanism anyway I think we can through this mechanism make our government more open more transparent and more participation thank you for your attention now Audrey would like to answer your questions so don't hesitate to raise your questions she put me a lot I'll just scroll like three pages into the appendix thank you appendix this page another one yes so that is the more detail as you can see a lot of people feel that they would recommend other people this is why we can get viral for this platform because people really encourage other people to join only about half the cases that they participated that they feel that there is a meaningful change because of the petitions or because consultations and that's the experience that we have subjectively to of the workshop that we have convened but interestingly only a tiny minority have improved their trust in the government after using the platform and that is one of the KPIs that we are measuring year after year to see whether this actually improves over time so that's the numbers we have yes you think that's bad I'm saying this is the baseline has the number of people who have less trust is there a bigger number that take now that I use the platform where I have less trust in government no so this is a simple question like whether you have improved your trust in the government after using the platform I don't think it asks whether it decreased their trust in the government after using the platform but maybe we should ask that too okay if you don't have any further questions I will invite Patricia I have more questions can you you said some of the proposals were rejected can you say more about why and was that accepted the idea that they should be rejected because some issues is about political issues like national defense so if it's a president's purview then it's not admitted it took us quite some time to establish a boundary between the president's office which is about the military diplomacy and cross-strait issues and we don't deal with that for this platform now and actually we changed the regulation to explicitly also rule out things rooted in fantasy or science fiction because they keep to people keep having petitions for our ministry which the Raylian movement to have a UFO embassy and it's not strictly speaking the president's purview it's actually constitutional but we don't actually think it's a very good use of people's time even though people keep sliding the Death Star petition in the with the people and platform and say that we should have more humor but in the end we think it's better to save the public service time for ruling out purely fictional petitions are there any examples of this tool or your office at the local level or at the municipal level within Taiwan great question the joint platform currently is used by the administrative branch also the corrective branch this is a in Taiwan's constitution there's also a branch dedicated for auditing and they use this consultative platform to ask the people of a certain city that like the city government is doing something that they don't know how to audit but instead of calling a stop or pulling a break to these things they ask what people's doubts are around there's new ways of doing things and they collect their doubts and ask the ministers in charge sometimes me if it's about social enterprise and then they establish new auditing standards I think a win-win solution for everybody involved because it basically translates people's fears and doubts into new auditing mechanisms that enabled the administration to go ahead with innovative ideas and as for cities and counties at the moment five cities and counties are in this platform as well and usually just for proposals or consultations but for Taipei city it's special because it's part of the iVoting process so the petitions on the Taipei portion of the joint platform actually gets put after a threshold into the iVoting stage where it actually becomes binding to the administration but in the city government so far only Taipei city does that and I think in a few months we're also going to offer participatory budgeting support for the city government so they can also use join as a common PB platform so in that mode we'll have a lot like council in Madrid and follow up do those same five cities offer some type of 311 or non-emergency like public ticketing system? yes they do and also a follow up I'm hesitant to say this with counties people in the room but also another thing is before the federal government started working on this a lot of the work was being handled at local governments and it sort of grew up out of that sorry was that a question is it coming yeah are there examples of when elected officials at any level of government also participate in this as a way to engage with and discuss with citizens or is it primarily for citizens to interact with government like I'm just trying to understand like how to weigh some of the deliberations are yeah it's a fair question some MPs use this platform as a way to kind of test the water of how popular their proposals are some parties use this as a testing ground before they bring it to the national referendum platform we see a lot of that dynamic this year now that we have a really binding national referendum act so yes it's been used to all sort of political purposes what are the are there any like what are the amplification mechanisms used within the platform to kind of highlight things that people should be focusing on and is there kind of a email SMS based communication scheme and is that done in kind of an automated driveway or is it done in the fun narrative way right so there's a weekly digest that I receive every week because I subscribe to every petition and regulation category so I receive this very long email but that's just me and actually the NDC people called I think almost a thousand NGOs and CSOs in the country that are registered as interested in specific areas so that in a privacy informed consent way their address, email address is used after calling them and confirming to automatically subscribe them to the bulletin of the petitions and regulation pre-announcements that's related to the CSOs stated mission and they get those letters for free and also people who have participated in the petition get those you know Amazon style implementation you may also be interested in some other petition and regulation that's I guess the more fun part and also all the petitioners and counter signature people are required to have an SMS number mostly to avoid you know people registering 5000 emails but starting next month I think both email and SMS are required so that we can send those letters through email and what do you use to I mean I guess the SMS goes a bit of the way but how do you make sure people are in Taiwan is that a concern that people are of any type of status? Yeah if you're a resident actually you don't have to be with the you know a full citizenship that's actually one of the key changes to the regulation so this is unlike the referendum or the normal bulletin system where you have to have full citizenship all we ask is that you're a resident that you're staying here and is that voluntarily enforced or is that done through? It's voluntarily enforced but I guess you know just during the time to obtain an SMS I guess you can work around it's variously using prepaid cards or whatever but you do have to take a checkbox that says you're a resident or a full citizen What if you're a citizen but you're not a resident in Taiwan? Then you're eligible so you're either a citizen or a resident Patricia it's your turn Okay Thank you Okay Thank you Tiffany I made a simple PPT for you guys and let's start right now but I have to say first I never go abroad for studies so my English is bored and I learn everything from Hollywood movies so if I say the wrong words please tell me Thank you I'd like to share about what our PO is doing I think it's all about unleashing the challenges inside the governments especially for our people like me the civil servants Okay This is Fang Rui and Tiffany just mentioned but I think it's relationships like an inverted triangle in the other part the left is led by Audrey and the left part is contributed by NDC to build the joint platform, e-platform online and what we PO Network do is basic on these two parts became a golden triangle and we I think we really create a new way or I'm not sure if I can say it as a model because it's very new and young so I just want to say we create a new way to serve our people Okay sometimes I ask myself a question what's the difference from the tradition way and the new way let me borrow political system a political theory that David Easton he just he said political political process is our system and political system theory that officials doing is to design and to execution the policies and to evaluate the effectiveness or efficiently efficiently or not but you know there is a left part about the system is black box some people always say the process of policy left a big black box that no transparency that people outside the policy we don't know what happened in this black box so I think the main contribution that the network doing is to take of the top cover of black box for political system and what the people always doing is we use new technologies tools and a new skill to bring multi-scale holders together and to develop solutions or improve we do is propose online people propose online and second the motion online and we made a collaboration meeting face to face to get different interests together and according to join e-play we use the new tools just like join e-platform and or we broadcast the live show with collaboration meeting live stream and I think collaboration meetings make it more transparency the other question comes what makes us become a PO many friends or co-workers always ask me how to become a PO or what kind of personality or capability makes you become a PO I'm not sure I can answer this question properly but I want to say something just just one time I think about even low level civil servants has his or her dream to achieve that is to improve the quality of policies and make the old stakeholders be more happy let's sometimes people said when you become a senior public servants you will lose your passion and work is daily life nothing new but I think to be a PO is a new opportunity to review yourself and to check out how many passion you have and you may contribute to other people and our job is to as Tiffany mentioned before we check join e-platform every day to collect established cases to dissolve we open meeting information and records online as we have to do and we join collaboration meetings lead by PDs or sometimes some issues didn't enter PDs pick to open collaboration meeting so we make it ourselves inside our ministries and we integrate inside and outside units actions of the ministry this is the most complicated part of our jobs because in different ministries some PO things they are doing meaningful jobs and some PO sometimes think they doing their jobs in very tough situation that's different why empowerment or delegation is in a different degree or situation so but I think I'm so lucky that my boss give me totally trust okay on the PO network we have the structure and hierarchy first level is Executive Yuan PO board I call it lead by Audrey and the second level we call the ministry level just as Tiffany said we have 31 ministry and got almost over 6 PO and now we are encouraged to create our own PO system under our ministry so I'm lucky to have my boss sport and we just publish administrative rule as legal basis legal basis of our works in the E platform so many people concerns about the agriculture questions but what is the FAQ in agriculture field on joint let me tell you the first place is about the food safety especially Taiwanese is highly concerned about the limitation or the regulation of a pesticide to use on the vegetables or animal products so it's like that and this kind of issues almost related to ministry of health and welfare so if this kind of issue comes out and we always have collaboration meetings with with the ministry of health and welfare and the second place is the struggle between industrial development and environment protection this place I use the industries not means the industry you know what I mean is about farmers or the fishermen and these kind of industries the Penghu case we mentioned earlier is about the fishermen their right to fishery and between the proposer is several diving lovers but they are also the warrior about environmental protection so it makes this proposal by good will but it makes the fishermen's community feel anguishes or nervous worry about their life or their living is to be harmed so it's a special case and the third place is about the animal protection it's very interesting that animal protection is a new booming issue in Taiwan we know the they are community with members not too many but they speak louder and in the internet so they are so solid and they can make proposal one didn't pass the review and the second one is coming following so close so we always deal with animal protection issues and the fourth place is the disease control via animal rights of their flu is a very severe problem in Taiwan so we always see in that kind of issues in joint play e-form okay this is the Penghu case I think I don't want to waste too much time to explain it but I want to share to separate the rival rival teams I think I don't know how to put it but I just use this term that we said two rooms one for collaborating meeting in the other room playing the live stream for online participants and the people which came to show their highly concerned and I'm so glad that Audrey with us in the second room and to let Feng to host the collaborating meetings the multi-scale holders came together including public sectors and the local government fishermen's association and hundreds of residents and some local councillors they are just the people who protest because they're trying to play some roles let's Taiwan's democracy okay let's the slides I want to share about the challenges we met we found that when the proposals are clear and the stakeholders are countable it will be easier but however the reality is always more complicated sometimes we have to make some pre-meetings to clarifying what's the real core problem because sometimes there will be some noisy signal in the proposals you cannot clarifying and the second is to transform the organization culture is quite a tough mission to extend the new methods requires more efforts because in the civil servant system people has habit or his way to doing things so especially the young generation can learn the new technology or new skill quickly by the senior or the elders one they just don't like to or not used to do this so I will organize some training and to asking for help from the PDs and the third is PO have to urge themselves all the time not to often the existing common chain inside the government the empowerment is very important because sometimes we have to host some discussion or pre-meeting but it will often the existing common system when we deal with animal protection issue I'm the director of secretariat my daily work is about liaison to media and congress but animal protection or livestock is not my expertise so to deal with this issues we have to invite people who in charge with this problem less the unit about this part and if we discuss together who will listen to or who's the boss it's a quietly a little bit of how to put it a little bit hmm huh? really delicate so you know they sometimes always don't like to create a new battlefield and they will always hope you just go through and close the case everything is okay and everybody is happy but the value of PO network is to push our insiders to face the real case that we have to and we have the responsibility to solve so we can let happen so sometimes we have to work together but I have to urge ourselves not to offend them and for there's not every collaboration meeting will lead to change of rules but sometimes in the happy ending we don't have to do the lawmaking maybe we just modify our executive process is okay can cause a good end but sometimes we have to relocation relocating some resources to modify or to fix the problem and sometimes we have to launch the making process it's different but always it is still I want to say means a lot about improving mutual understanding about among the people with different interests and last thing I want to share is in my opinion maybe our ultimate goal is someday there is no more P.O. because every single public servant should have capabilities to internalize the value and the spirit of P.O.'s role so we have a far way to go further but we will try our best that's all I want to share with you thank you so much thank you so much can we get applause for P.O.'s today amazing so we are going to wrap up now there is a list of resources over there the main one is hack folder that we have been talking about throughout the evening today and just to give you an idea of what that looks like here let me do it this way so hack folder is a new tool that's been built by GovZero and it's an easy way to like share links so all the links, collaborative notes, timetables all the slides everything is there under that URL we'll email all of this out to the folks that have registered so you'll get it and we're also going to be emailing you two feedback forms for both days we'll email all of the ones that correspond to you in the interest of time we wanted to take a really quick group photo if that's okay and then I'm going to pass this to Devin who has one more announcement for everyone only one more announcement so after this event we are going to have a reception it's a or maybe a hundred plus people of RSVP'd at this point it's going to be a big deal really modularized space so we're going to need volunteers between the hours of whenever this ends and the beginning of the reception that was advertised at 6.30 to help explore the implications of these crates these doors being movable and how things can get cleaned quickly and enthusiastically so after this picture which is going to be a lot of fun you can come and see me and I will task you or you will peer task each other coming up with some methods to clear all this, make this prepared for a presentation that's actually going to take place I think at 7 and the reception and the presentation Audrey's going to have a presentation they'll be kind of a bit of an overview for the new people who are going to be showing up so they understand what we went through over the last two days the fabulous experience that's a place and then yes and there will be drinks and a period of time as well into the evening so I'm here I'm wearing grey the things that you can do to super help us clear your stuff off the tables please as quickly as possible and if folks want to help take tables we can take them down those stairs and that would be amazing and layout chairs that would be upstairs tables upstairs but first we can get everyone up here for a quick group photo DJ wait what so we'll have all of the ad rooms assets in the photos everybody, everyone's going this way yeah but all the room assets will stay on the back yeah oh that's going to be funny all people in the back, all people in the front you know if you're tall you know if you're short we're going quick so can I ask you to take a photo of three photo of us yeah we've got a 10 second timer and we've got five shots okay so okay let's see let's see yes okay okay oh he wants to be tall you've got to be able to go on too yes any suggestions and any we'll make everyone smile no yeah I don't have any agency you have an agency when you push the button okay well is everybody ready to smile for 10 seconds here we go alright something happened yep no blinking it's an exclamation point really our customers talking about shooting movies no hold on actually I didn't see anything well I put it on film not your camera my camera is on yeah so I think I put it on we can just do something kind of separate something like here we go try to practice their natural smiles really practice your responses mark what 10 seconds alright alright I'm going to push the button and then we can count backwards from 10 together alright button push oh yeah yeah 5 8 something happened congratulations alright tables cleared tables cheers stay tables upstairs thank you so much everyone for attending and a big big huge thanks to Prime Produce for hosting us here in this fabulous space yes and just to be clear the plan is you go eat we'll come back reception overview we're going to help volunteer clean up so that the reception is awesome come see me thanks yeah this is just a reception the assets of what we put worked on today right now hurry up and get the fuck out of here