 So, thank you for being able to be interviewed by me and yeah because I just had a talk with Anders and he found that with the usage of algorithm in public participation it's quite advanced and quite new in digital geography. So he wants to understand a little bit more and I think that's probably one of maybe it has potential that I can write a chapter around this because there are loads of debates about whether algorithm or machine learning this kind of technique has its own agency in making decisions or how do they participate or collaborate with human actors in public participation process and I think with the use of police is somehow very interesting and experimental so that's but we are from the more geography background so we both us have some doubts around the algorithm and we want to kind of um black box to you know to understand and I think the knowledge could be shared by other researchers and so that in the future maybe there are more much more discourse or discussion can be established. So thank you very much. That sounds like my pleasure. Can you see my screen? I'm supposedly sharing my screen. Okay great. Yeah so the big question here is as you said the space itself behaves like an agent in a discussion and so yeah I think it is when we say we use polis to automate a way facilitators job we mean mostly that it does things that you know human will necessarily do if we announce the the methods beforehand and in the sense that it's predictable but or explainable as the jargon currently says so it is not strictly speaking artificial intelligence in a sense of deep learning but rather as in like really handy automation which is supposedly easy to explain and we can of course demonstrate it by having me explain it to you yeah and right but but yeah it is not deep learning and as you said it's a it's a marketing choice choice that Colin and friends you know call it AI powered conversations but that's a separate conversation you know between between you and Colin preferably yeah so let's get to your your questions yeah. So my first question is I have some thoughts about the clustering I'm not for sure if it's called clustering algorithm but so according to the first conversation that we had you explained the way that how you choose the most representative common within one cluster or across different clusters but I don't know how do you break it down into clusters. Right right so as you can see here at the polis math repository there is very simply to say a maximum number of clusters which is five okay right and then the idea the intuition being it is a k means algorithm with with k being anywhere from one to five and it recalculates the k value which is the number of clusters every time a new vote or a new comment comes in but it doesn't suddenly change the cluster number for UI reasons so there is a buffer for example there's three groups at the beginning and then after a while the system think it's better to have four groups but it will not change three to four until it's consistently four times four that is to say after four actions it remains the ideal group number remains four so it will then change so there is a buffer if after two or three votes the system think it's you know it's better to to go back to three then the user doesn't see it's suddenly splitting and suddenly joining it is I think mostly a user experience thing now how does it think whether three is better or four is better it is a standard you know algorithm to choose the best k so it use a idea called silhouette and the silhouette value here it is a value from negative one to positive one to determine whether a group is a good fit that is to say whether the points in a certain group or a certain cluster actually belongs to the cluster by calculating the distance between them and it is a very standard algorithm and so if you check the the Wikipedia article on determining the number of clusters in the set I think it uses the standard silhouette method to try to partition it into two three four or five groups and then see whether any of these k values yields a better silhouette number for all the groups involved and then it just chooses the one that's best so so it is actually quite standard statistics speaking okay so and so the way that like the the reason behind using these algorithms are they are any particular reason that you choose k means algorithm and the studio eight method well first I didn't code the math part but we can simply say okay it is mostly Christopher's decision and so yeah to to have a in-depth discussion of which methods did Christopher evaluate you have to interview Christopher instead I think personally I find the k-means and silhouette method easy to explain but I think it's it's the main attraction to me but because I didn't write this code I don't really have any intuition on whether there is a even more explainable method that's around right okay so I think perhaps if it's possible I would like to also interview Christopher or calling in the follow-up yes certainly certainly yeah under both the marketing and mathematics department okay right thank you and so because I've I've checked Wikipedia and they say normally clustering algorithm is a set of loads algorithms so when you when you say you use k-means algorithm it's not only one one algorithm but it's like a collective it's my understanding all right or just a bit curious about so that's when you show me earlier the screen is is I think that's the parameter the setting for the formula or the algorithm that you use but I wonder where I can see all the algorithms that you use in police oh yeah certainly it's it's in one single github repository just called polis math yeah I went there but I am I don't like if I type like algorithm and then there's nothing like I wonder what's in then that you use so that I can search by myself or okay well I mean it is written in in closure which is list which is not exactly it is very mathy but but it's not very easy to follow if you are used to to more um let's say object oriented way of programming but I think most of the algorithm you're interested in are in this one single file called clusters oh okay so it's just cluster.coj in polis math and I think it is really pretty standard iterated k-means clustering I think this is what you're looking at is like the entire algorithm here yeah so I'm sure that you can dive into it and find a hyperparameters and parameters to this algorithm but I think they the polis team did not deviate from the standard statistics or mathematics at all when encoding this part right okay thank you that's that's very I would say that's very interesting because there's another group in academia talking about so there are loads of algorithms that have been used in our lives in our everyday life but so forth like socialists or social scientists uh we don't really understand it but we we want to understand it in in the amateur way or in a layman way so that we can share and we can explain it in a more easier way to to the common people or to other people who are interested so that's I think that's you know even though I'm I can't code or I you know there's some barriers but I still try to understand I think that's that's the starting point and thank you very much to explain all the things to me right and and there's intuitions like like this kind of strings just after each function so after right after the definition line for a silhouette it writes in English more or less what it's trying to do in code afterwards so so yeah it should be somewhat helpful if you just want to walk through this entire file right thank you very much I'll definitely check it out um so second question is so I wonder when you decide to use police to facilitate all lose confidence I wonder what do you expect what kind of result or what kind of effect did you expect to get from police right so when we start a police discussion as we previously discussed is usually because the sheer amount of input comments exceeds that's the reading comprehension time for for everybody involved and so I think the first effect I would like everybody to have is to have an overview of what kind of arguments are there because it is easy to overlook that there are better or more interesting arguments if one is just focused on one particular conversation threat and so police I think the the main effect I want to give is more or less the overview effect which is the effect that let people see that there is a variety or a we should say a distribution of all the possible ideas in an idea space I think that is the first thing that I would like to people to consider okay and I wonder so for some particular issues such as Airbnb or Uber why why did you choose to use police or not this course in the in the questionnaire or you know giving a big picture stage no but but I think I already answered it is simply because the incoming opinions are so much in volume and in diversity that it is practically impossible if had we used this course or a traditional foreign method for everybody to read it all and even when people can read it all they would not be able to tease out consensus from the the sheer volume of opinions and the same thing happened in the drunk driving case well in particularly the you know the we already know that there are people who are interested who will iterate the same opinions over and over and in polis people who share the same idea would just consolidate into a single single dot so they will not waste everybody else's time by revisiting the the same points over and over rather we reward consensus making behavior yes I think my question is in v Taiwan there are a number of issues and I found some of them and you don't use police in the questionnaire stage right but but but police so I wonder is there any rationale behind it yeah we we usually use polis if we think that there are a lot of opinionated members of society who will want to to come to this space we don't use polis if we think that there's not so much strong emotions or feelings or reflections and the volume is something that a forum can handle or a questionnaire can handle and then we use forums instead so basically we use forums because we know that we can moderate the forums ourselves but when the volumes of variety grows to such a point that human moderation is no longer feasible instead of giving up we use a automated moderator that's the intuition okay I'm just curious how did you know in the very beginning when when you start to collect different opinions like before before collecting the opinion how did you know this issue will be very popular like most people will participate well well you can you can analyze the this topic and see whether it is hot you know in in social media in traditional media whether it is being actively followed or you know things like that so so yeah there are ways to analyze the traditional and social media to see if any particular topic is likely to to get people who are very opinionated or a lot of people and for things that the government or the society doesn't quite have this strong opinion on we don't actually have to handle this volume of input okay I see your point so that actually okay brings I want to skip the third I will come back but I will go to the fifth and the fourth so the fourth is so when you say there's a moderator in the discourse in a foreign format that's right and and then I wonder so perhaps there is a common moderation in police there's like a by default setting such as lazy or strict I wonder which mode did you choose well we we tried both ways I think um initially in in the uber x case we we we did strict moderation but I almost watched it constantly and approving new comments as quick as icon and afterwards for a bmb I think we went for lazy that is to say pass by default moderation which created a a explosion in in number of comments and the comments it's not um very high quality at that so we had to to uh very quickly improve the comments um as billy did uh during the drunk driving one so um I don't I don't have a very good rule I think um if the moderator can be like 24 7 uh looking at the moderation then strict moderation is good because it doesn't waste anybody's time but if the moderator can only visit a few times a day then perhaps lazy moderation is good um yeah I don't really have a very fine heuristic on this but it mostly a trade-off between the time wasted on the participants and the time spent by moderators okay that's actually quite interesting so you were the moderator that's the strict mode for a bmb no no no for for uber x and for for a bmbi was also the moderator but it started in lazy mode so are you the only moderator well yeah polis polis at that time only allows one moderator there's no group moderation and it's very recently that v taiwan created a group account and that we um you know made polis run independent of hiroku so we can share a group moderator all these are very recent advancements uh and so um every case before like um this year uh there's by necessity only one moderator wow you did lots of jobs lots of tasks and I I'm interested so that that's very interesting I wonder how did you how did you moderate what in a strict moderation mode like how did you approve each comment like you know is there any product protocols or principles behind it but well easily if it's a duplicate if the point is made by um strictly speaking um you know a a comment to address exactly the same thing or something that is um you know very high correlation with this thing that I tend not to let it through uh but if it's making a a new point or a point that's that's not equivalent although somewhat similar to a previous comment and I let it through it's mostly just the duplication okay but so you have to remember like you know when the amount of common goes really huge you will have to remember like oh god they are maybe 100 comments which are very different from each other so when there is a new comment comes in you have to remember oh this goes into the first one or that that's right that's right but but I think really substantial comments uh they tend to resonate already with one particular group or the other so so it's not that hard um I mean we we already have to do this if we um are doing a face-to-face deliberation uh if people bring up new points that's already covered in the um handbook or or in any of the stakeholder materials then you have to understand and remember that so it is um I think easier for a facilitator to to simply navigate the comment space just as we did with real-time board it's certainly already easier to simply see one incoming comment and and remember whether it's a duplicate than seeing a one incoming comment and think oh which group we'll tend to agree with this which is what a face-to-face moderator often have to do but by by leaving that part to algorithm we can just focus on the quality um and diversity of the comments so yeah while it is work it is already a significantly less work okay that's that's interesting okay so go back to the third question uh I wonder so will police ensure every comment is viewed by more or less the same amount of people to to make it fair when when they single out the most representative comment in one group yes yes yes it does that uh but but we are also talking about a algorithm change uh recently uh I think if you interview Christopher you can go into depth uh in that that there is a tension between wanting to give every comment um the same number of um votes um or or eyeballs in in order to make sure that they get fair treatment versus making the um time that's spent on polis um most efficient use of a voter's time if there is say dozens or even hundreds of comments um a a random distribution will mean that the quality of the comments seen by any viewer um is by necessity not very correlated that is to say how I vote in my first comment or the second comment does not really um give any indication of what kind of comments I will see on the third or on the fourth uh vote and that creates a um somewhat different experience compared to uh what we are now thinking about which is to give the voters the most controversial um comments first so that and after two or three votes we can be reasonably sure uh which group um that this person belongs to and then we give them uh as fair as possible um the comments that this particular group um tend to be divisive about in order to further um refine the the comment space so there there is an argument to be said by by you know try to use um the existing votes of one person to inform the next question or comment to show versus the current way which is as fair as possible to give every comment the same number of eyeballs uh we haven't really did a do a uh empirical trial of the new um let's say it's more um narrative-based um um way of presenting comments so so I can't really tell you how the experience differs uh but the design um intuition is so that people will become more engaged because they get more relevant and uh controversial comments early on and it engaged them into constant thinking um based on where they are but uh of course I can argue the other way which is uh they don't get exposed to um things that um everybody um you know tend to vote yes or tend to vote no and these are useful too because it um it established a a report or a um understanding that um people can come to agreement even if they have controversies but by having the controversies upfront it may create experience that emphasize the controversies so I don't really know because we haven't made a field study yet okay I think that's that's a very very interesting argument I'll definitely follow up with uh Christopher because I think that you know sometimes you will see because of these um mathematics or algorithms statistics behind the they will they will slightly or or subtly change the way how people participate or even change how they feel like you sometimes they will they will get very sympathetic because oh you create another um button say oh you can say yes or you can say what so I I think the emotional changes will be quite that that's very interesting okay uh so and so that comes to the fourth question I wonder um so what have you tried to get a better performance out of please for example I checked the the document they recommend every user they want to please to to leak for example to leave c comments so that to make sure there are some diversities in the very beginning or um other yeah we definitely do that we definitely do that uh just as a face-to-face deliberation start with a informed phase where people get a handbook that lists the major arguments and positions from the relevant stakeholders we try to put um as succinct as possible the various positions um that we get from the stakeholders beforehand and get them into seed comments uh we used to have this formula which we still follow more or less um not religiously um in that we we have nine seed comments at first and three of them are profiling questions that is to say in the uber x case there would be uh do I do do I have a professional driver's license if I use uber before and things like that and three of them will be uh position questions uh I think taxation is important I think insurance is important uh I think registration is important and and so on and and three of them uh will be um like wish uh questions like I wish the government will do um do more uh to to reign uber x in and things like that and so yeah I think the initial seed questions are meant to let people know that there are good answers both the s and o on those uh profile questions which means that there are people who are like them and people who are not like them and the positions are necessarily uh follows from the um you know from the profiles but not always and then the wish uh or the statements that as the government we have received from the lobbyists we also try to put them into seed questions because after all that is uh what people consider important enough to to raise as uh lobbying statements uh what we want out of polis is a refinement uh out of those uh very strong um or very polarized um positions so so I wonder where did you get those um seeds the the question for the seeds from we we asked the stakeholders like if there is one thing that you can uh put in as your wish or your position what would that thing be and things like that so can I still find the archive the questions oh yeah you you suddenly can uh I have a slide uh that shows exactly that I think I even recorded it as a um as a kind of online course to how to use polis so I'll send you that afterwards yes yes thank you very much and so also in in that document on keep keep book it also states that um there's a limited number that um it can only have less than a thousand comments per time but sometimes because uh vita one is quite popular so most for say uh uber or airbnb there there are more than a thousand comments and I wonder is it kind of uh pushing the the capacity or the stress for the police or it's not a concern of yours well if it is things like uber x or ambnb people tend to be willing to spend a lot of time on polis even answering all the uh comments and so yeah it is less of a concern but uh we we do think that for more mild uh topics where people feel less strongly about um then it may be good to moderate out most of uh the the repetitive comments and and or introduce uh a different presentation order where people are presented more controversial comments first as I mentioned to you that's Christopher's idea so so yeah there is a certain guesswork we have to do on how much time uh any voter any participant is willing to spend on polis and beyond that number um we we will see that the sampling gets less and less effective uh each comment has less and less significant participation and so we'll have to change accordingly but for the really hot cases like uber x and airbnb I am not that worried okay okay I see uh right that's that's that's fine uh I think we can go to oh and this this is a quite interesting one the last one encourage participants to to stop back a few times to vote on new comments which are only showing you up very very late but recently or or kind of you don't let them to come in at the end because if they come later then they might not be well it's kind of comes comes to the first point that if they are all being voted by a certain amount of people yeah if they leave their emails they they'll get remind us that's there's something that polis already does but I think this allowing new comments toward the end um may be useful but but you may not I mean um in airbnb's case we get like two days before the the the polis period draws to a close there's a huge influx of participants because airbnb send an email to all their members in taiwan asking them to come to polis and so so if we disallow new comments at that point we will not get high quality conversation despite there there being only like 48 hours or something left they still have a lot of I think very useful interactions and we can always extend the the date I mean if we don't get a super majority or if we don't get resonating high quality comments we can always say okay we run it for another week so so it is not a hard deadline well that's that's quite interesting I wonder how do you think how do you judge if it is a high quality conversation well it is a high quality conversation if we manage to get people from different groups to nevertheless agree on something that's the whole point okay so the whole point is to get a consensus out of the participants out of different opinions that's exactly right and and by consensus I think it was Christopher who came up with with something like this which is which means that you know there exists a certain common C which for every group in G where there is a majority of people you know and this is basically saying you know anything less than 50 percent is treated as zero right and and then time to two so that it still normalizes to 100 percent and it multiplies its support across all groups and the higher the better so the idea is that if there emerges a common C that is agreed by everybody in every group then of course it has a perfect score but if it's agreed by all groups except one minority group and that minority group less than half people agree with it then that comment is zero actually so it has to get more than majority support from all groups in in order to even have a non-zero number of the consensus function and the idea is that if we have a handful that is to say more than three I would guess common C with a high enough consensus score that is consistent then I think the policy conversation is a success otherwise we may have to run it a little bit longer that's very interesting so the Christopher wrote come up with this formula himself the consensus formula oh wow okay that's brilliant that's very interesting I'm definitely going to talk to him and so next so um oh I wonder so basically you have the forum tools and a police tool and then to use this tool you collect you gather different ideas or the direction of trend with the the issues and I wonder how how do they been integrated uh because since you have so many different viewpoints and how how do they being gather or bring to to the other stage what were you having the the expert meeting the face-to-face one right now it is it's simple right we just asked the police algorithm to come up with a report and to read from the report but before we had a report we had to do something manually that is very much like the report only much more tedious the idea is that we first list the the consensus arguments and then we try to find in each group which are the comments that are representative of that group meaning that the comments kind of defines that group meaning that this particular group agrees on this but pretty much nobody else agrees strongly and so and and that's it that that's all we do so we let the face-to-face expert meeting or we let people online know through a presentation how many groups are there what are the majorities consensus arguments what are the non-consensus but nevertheless distinguishing sentiments within every particular group and that's pretty much it okay so okay so when you are facilitating the face-to-face expert meeting yeah I start with a briefing of the of the police face I wonder would you so there's a minority opinion which was still in the presentation but I mean during the discussion with you know experts all kinds of stakeholders uh when you as a facilitator would you how did you treat the minority opinions in these conversations or discussions well if it's a minority group one of the telling points of polis is that it it doesn't ignore minority groups at all right so the minority group I would say okay this group only has 10 percent of population but it brings a very good point which is blah blah blah but if it it is a comment that nobody agrees on like a negative consensus then maybe it's just a really bad idea and and so I I don't actually present those and unless it's it's significant in the sense that you know the reverse of it it's still a useful sentiment I remember I picked one from the uber x case which is to say we should not have this conversation we should just find uber and and a majority of people disagree with that and I shouldn't you know strictly speaking show it in the presentation but I think it helps the discussion to to let people know that you know most people don't feel that we should stop the discussion we should let the discussion go on so it is somewhat arbitrary I admit but otherwise minority groups get representation but not minority comments that are ignored by everybody okay right okay that's okay and okay that that's really interesting and okay I've heard from one of the colleagues who said that there was one that police was shut down and it was at the end of one of the issue which he forgot and I want to follow up do you know what what happened and what caused it to shut down I don't remember anything like that I I remember that a report function is broken when we want to to get the drunk driving thing and so Billy had to do it manually but I think that morning the report function is repaired so we were able to include the report functions output the PDF file in time but we didn't have the report function anyway so so it's not like that right it is the first time we tried reporting function so so that that's it I think that's all it is what you say you don't have the report report function is that because you use police as sars so that no no no the reporting function is a new is a new feature it didn't it didn't get completed until I think around drunk driving in NCII so when we were doing like unlikely curcells or airbnb or uber there's no recording function to speak of oh okay so why why do they want to have this function well because it is what everybody asked them to to write because it is tedious work that all the facilitators have to do and there is no human creativity in it which makes it a prime candidate to be automated yeah so do you have they still have the spreadsheets or the sheets where you create those tedious jobs that I can take away there is a there is a github repository just called polis tally and if you look at polis tally you you probably have all everything here like like you know so this is the polis conversation and if you search for the first nine comments you will see the c questions that I just talked about I am a taxi driver I'm a uber driver I think insurance accident insurance is important taxation is important conflict resolution should be handled by the ministry of transport and communications I think surge pricing is fair I have used uber I think multiple dispatch systems can be allowed on the same taxi I think there should be prominent display on a car that that runs for service and I think to run after such unlicensed operation is the MOTC's duty and and so on so so this is the basically the matrix that I operate off before the reporting function and this is sorted by consensus by the way and so you can see the highest consensus is really pretty high wow the 97 and 96 respectively wow but yeah that's that's like so you said those are the seed common or the this is the final result sorted by percentage of consensus I was just uh searching for single digit uh index which means there are seed comments um because they're first nine right um and so to show you and so of all the seed comments uh only I think the the accident insurance one made to the like complete consensus list okay only this one yeah I think only this one made to the list so the conversation actually grows uh more than the seed comments yeah and and this is what polis is why we use polis otherwise we just send our survey right yes that wow that yeah that makes lots of sense great uh so see okay next one uh how do you ever use because I I I seen on keep hookbook they say there's the data export function um how do you ever use the data of course this this is what you're looking at is the data export ah yeah so so uh of course I I write some programs to make it easier to see um the raw export is something like this which is not very easy to comprehend uh yeah but uh but it is actually um easy um if if I explain a little bit every comment has a body a index number of a green number of a disagree and a percentage right and then for each index you can match it with one voter and uh this voter participant one uh belongs to the first group um the k cluster the first cluster it posted no comments um and um they voted for 29 times agreeing 14 times and disagreeing 10 times and then for each comment that are there there's almost 200 comments here is how they voted one is a yes negative one is a no zero is a skip if you don't see anything um that person hasn't seen this comment before okay wow that's that's very interesting so is this also on keep up that this is on github if you search for polis tally uh all the polis we have run I have um dumped its data and committed it okay right that that's that's that's range so would you be able to know the part what the geography of the or the gender or age are the profiles of the we don't ask that questions so we we don't ah because um in polis uh privacy policy oh yeah it say that um they might possibly also collect the data uh which is when you register uh either through google or for facebook through twitter whatever yeah that's right if facebook choose to reveal your your your agenda uh it is plausible that people who sign up through facebook you can cross analyze its profile but uh although technically we can do that um now with our own uh polis login and our own uh non-hero conversion of polis we haven't even begin to think about doing it um although yeah now that you mentioned it we can do this technically but we're not doing it now okay because because I wonder do Christopher or Collin do they do these analysis because the the policy the privacy policy looks like that they are going to do some cross they do do geography analysis I remember for the uber x case they correlated the number of population in taiwan and the number of participants of the uber x conversation uh in taiwan and they found that there is um very even there is no um over concentration on large cities so and which makes us really happy but I don't know which other analysis they do so maybe you'll have to ask them yes I probably need to ask them because this is quite popular like to yeah to use this data because when you have a conversation those of data are created accordingly or collected accordingly so that you can kind of recycle or reuse those data and yeah kind of like a data market I do agree it's it's useful it's just maybe useful from an analytic sense or academic sense maybe not not that's relevant to to the deliberation itself um I wouldn't certainly if I think the gender or age is important I will have they included as one pollist question um but but if if it's not then it's kind of kind of circumstantial because people not do not necessarily use facebook to log in for one and people's facebook profile may not be that accurate for another yes exactly yes that that's true um so next one I wonder all the data collected in the process of police who who has the ownership because I checked the privacy policy uh of v taiwan it is that most of the data it or it's it's you know from the from the cc the zero uh license that's right so it belongs to everybody or it belongs to nobody no one obviously in terms of ownership and that's very important when I checked the privacy policy in police uh they said every data every piece of core source are owned by police and subject to uh intellectual and proprietary uh rights and low protection so I wonder you know because you have used police in a quite interesting way in the software in direct way so do you have the control of the data like who owns this data well um the first thing is that um the privacy statement here um I don't think uh police says that uh your um IP address and things like that become their intellectual property um I think um no I don't think it says anything like that here so maybe I'm mistaken oh yeah but uh I think one of them um they say the for example the comment so when you use the user materials is it uh materials yes user user materials uh it's it it's it's somehow you you kind of um you will know it's you have a comment in a public domain and then they might use this data for further analysis and so that and also according to uh I think there's a intellectual is it intellectual I don't think there's a intellectual property I mean I mean of course by necessity just by showing your comment to other people you have to give police a non-exclusive right uh to show it to other people otherwise there is really no point of police existing um but I don't think um somehow you become a contracted author and you give the uh an IP right and you can't use those copy copyright to statement yourself I don't see anything like that here I think there's another part which is um to talk about a say to track the source code they say source code well this is the but this is termed use this refers to this refers to the the source code of police itself of course they will want to assert intellectual property uh on the police code because otherwise they can't put it and also I think it so express uh like several lines to to talk about uh how many with all these materials I think they use a collective word materials to express uh all this data and the the stuff are owned by police yeah but but this is material does not this is not user material right so so this is things that Christopher has written this is not uh what a participant has written and and so of course police will want to retain IP right because otherwise they will not be able to release it under a open source license they will have to you know check with everybody else so so I don't think those two materials are the same thing oh okay but I mean when you say that when they say there's a data so I wonder you know when these comments or all kinds of things becoming data saved in a data format and if they say they own this ownership and thought it's uh embedded um the Taiwan I wanted to do you have a say to use this data generated by police that I think that's just no yeah I think I think I think this is this is the scope is different right this talks about this service which is you can consider you know the inputs being votes and comments right and the output being a visual display right and then um police is essentially saying how we generate the output from the input is the intellectual property but it doesn't really say anything about the votes itself and the comments itself of course the votes and comments are under you know cc0 if it's a v taiwan context or whatever ip right is belonging to to the person who wrote this um and all the polis is asking is a non non-exclusive right for you to um you know let it show your comments to other people and I think that is pretty reasonable actually okay so Christopher wrote this uh the term users and uh no no the code I don't know which lawyer wrote this so so you have to talk to their lawyer but I think the intent is is never to control the intellectual property of user generated material I'm pretty sure about that okay right okay thank you um so according to GNU APGO version 3 uh if if you you know through open source you get the the source code of police and you modify it you will probably have to publish the modified police and I wonder does it does it apply to um the sandstorm police yeah of course um we will have to provide the the modified code to any user of the system that is actually a sandstorm feature you can whenever you use a sandstorm system download any data you have uh in the sandstorm as well as a copy of the code that is running um in sandstorm so that you can this is called portability you can move to to your own hardware anything that you participate on a sandstorm instance so sandstorm is designed with this kind of free software um copy left in cloud in mind in the sense that any user is able to obtain not only the data stored on the cloud but also a copy of the code that's running in the cloud and and so by that we I think we're already agpo compliant but uh in addition to that I think our modifications are all done publicly on github anyway so people can just follow our fork and you know chase the changes we made yeah okay well that that's fascinating that's very interesting and well next one oh did you have another meeting that's the three o'clock system I I noticed that the time is up no it's okay it's okay we can go on for another I don't know have an hour or something okay um so well there's the the the chief of IT sector in Taipei City Hall in one of the interview he mentioned that um well when in the government and outside the government say g golf zero when they're trying to build a software or a project they might have some differences the the value they believe or the principle they have so for example he from his perspective he thinks um golf zero when they're doing a project focusing on very agile and very uh specific and very fast and they want to be very effective but in the government maybe they try to say oh they focus on a stability and a safety of that piece of software or the system so I wonder when you are building because and because me Taiwan is somehow very interesting is somehow in between because it's a project uh proposed by in former minister in executive union but it's been built and maintained largely by golf zero so so so I wonder when you are doing this when you are building or set design the infrastructure of the Taiwan um is there any trade-off because you know there are some people coming from the government they might have different ideas of the design what the Taiwan should be look like or you know what kind of software it should be and compared to to you know you were from golf zero at the time and other people well I mean people who want control from the government side all participate in the joint platform instead so so I think the best thing for V Taiwan was that there is concurrently speaking a joint platform so that people who want government control can just put their ideas to the joint platform and leave V Taiwan to experiments um and on the other hand of course cybersecurity and safety is very important which is why we use um pretty much try and true open source components um and polis being the one exception because when we are using it it is not yet open source there is significant algorithm uh in it we don't necessarily have to trust their explanation to it we have to do a lot of validations and so on um I think see a guy recommended polis because uh first he nailed calling I mean like face to face uh and we want to create pressure for them to open source uh polis but before we can apply pressure we have to let them let the polis team see that uh really it helps to open source things if the large governments are going to use polis as part of the decision making platform and to get credibility you have to open source it if it is just a few hobbyists or private sector people using polis they may not care that much about algorithmic transparency so we kind of have to prove that polis can be used on a massive scale before we can convince them to open source it so we took a risk by using a proprietary uh SAS software as part of the detail and stack but otherwise we are building on very solid open source proven grounds that was very interesting I didn't know it is got zero uh CEO go and you to push them to open source uh polis okay I'm going to follow up with you that's that's that's a very interesting story and the the next one so um as the there are some discussion around city tag and the city well so that means say Desai Madrid a council uh to some degrees uh has benefited from say the resources from Madrid City Hub but outside that from media lab or from the very talented software engineers who are uh residents in Madrid or you know all kinds this this this city as a space for experiments all kinds of experiments and city tech would be one of them this this kind of discussion has been made in um well urban geography or in human geography at large so I don't I'm just curious because I I found out that oh say sunflower movement or all kinds of activity from golf zero even the Taiwan little hackathon all these hackathon they are all located and happen in Taipei I wonder what do you think of it is there is any connection or it's just some um accident happening no it's not an accident I mean in Taiwan we have 20 years of open source um movement uh with coast cup with um OSDC with the meetups with the various conferences inspired by coast cup and OSDC um so there's a huge amount of people who already are gathering in in Taipei um there are of course like Mop Kong and Gaoxiong and there's a bunch of people around NCKU in Tainan but by and large the the largest annual events in open source happens in Taipei and usually around academia Seneca because academia Seneca in the early 2000s has this national plan to push open source and it established a so-called open foundry to build infrastructure it's just like github before github existed to to basically give the developers not just a online space but also a legal space through the introduction to of creative commons and licensing uh advices uh as well as offline space that is to say the academia Seneca um building itself um provides a lot of practically free um venues for large-scale events uh graph zero itself um would have um not to be able to accommodate hundreds of participants if not for um scw's uh Chen Zhengwei's uh intervention that allowed Zhong Yunyuan academia Seneca um department of information technologies to serve as a stable venue spot and so on and so forth so there's many um offline space online space and legal frameworks this supports the open source community throughout the past 20 years and so graph zero kind of just rides on this uh wave there's many other teams riding on this wave as well graph zero is just the the civic tech part of it okay well that's that's very interesting explanation i'm going to follow with other people from graph zero and so the last one okay um so i had to talk with and tip into uh graph zero summit 2014 and he he said interview you and you know other key persons he thinks um the value or the logic behind these agile or anarchist or either decentralized value of v taiwan has some connection with the the silicon valley yeah certainly the the hacker culture uh as we call it yeah but but it's not really just silicon valley right the hacker culture is it's also mit it is also european like the chaos computing club it is pretty global but yeah it is it is part of the hacker culture so so i actually because that's his point of view and i i think it's more than that so i wonder you know what else is there any specific um cases or you know particular culture that you you make it as a reference to to to view it's graph zero or to to make it as the value of graph zero well i mean the leaderless uh values um i think it is part of the internet itself uh the internet is basically rewarding innovations without permission uh and we i think very consciously connect to the global open source movement uh which uh again after the invention of decentralized version control uh svk and big keeper and of course git now um everybody is free to fork any project in any way knowing that if um other people think it's a good idea they can merge it with minimal cost and so i think because uh both the ogao and i were both developers of decentralized version control systems we naturally took a lot of the analogy in the decentralized version control system culture and later the git culture uh into the formulation of the com zero ethos which is again um rewards forking and rewards innovation without permission but we also um has a lot of um experience with open space technology in particular the the bar camp and food camp um idea of open space technology or unconference uh that is of course a a pretty silicon valley idea the first bar camp uh is organized in social text uh a company that i worked with for eight years or something uh and still go for i don't know six years five years and there's various other gov zero um participants who worked in somewhere or another with the social text people uh and social text even in silicon valley is considered very radically decentralized uh and so there's a lot of social text culture in the gov zero culture as well but otherwise i think it is also very local the the taiwan people um already has a very active meet-up culture a very active culture where there's a rough consensus and just go and do something the organization management part i think owe a lot to the local mozilla community and the local coast cup community in particular um and so yeah you probably want to interview the ocf people who are involved with coast cup uh planning because the gov zero summit in particular is modeled after coast cup okay well that's yes i think that's you mentioned something very interesting because you know when you learn those with global experiences it's sometimes you have to translate or you know did you make it localize in in in a specific context and each context has its personality or characteristics so it's not that it's not that easy or it's not that say it's all the same so i think you know showing some character of the context inside one world will i think would be something i want to figure out and okay last question because i i i read some some texts on on the archive v taiwan they don't happen some history and so i learned that you use um you refer to regulation room and then as kindness like the prototype yeah we basically recreated regulation room uh using discourse and you know some get book and other technologies regulation room was written from trupo and we find its interface not exactly the best but the design principles um i think is really advanced and that there is something very concrete in its output as well the synthetic documents and so on so yeah we basically started exactly where the regulation room left off and i also think regulation room um leaves a very good trail in the sense that they work with ipm and other research institutes to publish a series of papers exactly what kind of training they gave to the online moderators what kind of preparatory material it gave what kind of interventions it had and so on uh and so yeah we also think that it's essential that the public servants involved um you know get a course on how exactly this thing works instead of just being another forum to attend and so i think having a concrete prototype as well as theory is very helpful so why why did you choose regulation it's because of course you mentioned there are some advantages to all these concrete profile they're doing they had they had been done by by the people from regulation by far it's the most advanced i i haven't seen uh any you know uh national level regulation making um that is has quite a success as a regulation room had um so basically we just picked the the state of the art and start from there okay because you in one of the the speech you mentioned there are some difficulties by the michelle has this professor oh yeah yeah that's one of the meta analysis that mentioned regulation room as well as others yeah he mentioned that there are three big issues they encounter so the first one right the ignorant ignorant wall yeah and then the second one what is the city wall that this is my translation and the third one is say too much information information explosion wall and as as three you know barriers they encounter and i wonder yeah i can get you the exact uh formulations in his paper later yeah yes i wonder when when you because you know they have some difficulties or you know exactly uh what kind of issue they have so when you are building designing the infrastructure of the taiwan and are you have you you know think of a way to circumvent or try to prevent these things from happen yeah certainly uh we we try to have a lexicon or a mini dictionary uh to make the the ignorance not that much of a problem uh we try to have uh dedicated moderators in order to foster a positive conversation instead of a toxic one we explicitly choose discourse because it has excellent moderation tools so that it's possible to edit away the part that is a hominem attack but still leave at the part that gives a substantial contribution um and this kind of piecemeal moderation is something that not many other systems have um and finally um for information overload um well that's where polis comes in but even before polis we try to um you know make final sub topics of each policy subject so that people don't go over the place but can instead of focus on one specific aspect at a time and so yeah we of course learned from the uh um advices and the issues that they run uh into so do you do you think one last one did you really think um because he he problem problematized the issue based on his experiences in the us in the state and i wondered do you think digital forum culture in taiwan also shared the same issue or there are other issue or you know in happening in the context of taiwan as you from your previous experiences well from my previous experience uh in taiwan we are lucky in that we don't have to advertise a lot uh to engage people in politics people are very engage in politics you don't have to mobilize them um and we don't have to think that much about rural areas and digital gap because you know there's broadband everywhere uh and so there's there's advantages in taiwan in that we can deploy really cutting edge systems without fearing that nobody will get on on them right so that's the the the thing we don't have to worry about but as for the culture of you know that the people who are the most ignorant comment the loudest the trolls culture um as well as you know a lot of uh foreign things of course it is the same everywhere because it is a property of the medium it is not the property of a population um because it rewards uh things that capture people's attention and so the attention seekers of course dominate the discussion and so again this is a problem that's also problemized uh problem ties by the discourse team which is why they call themselves the civilized discourse construction kit um because they find most online discourse to be not civilized and they try to invent or construct a system that makes in my civility and so um by using discourse we're basically um standing on the shoulder of the the troll masters uh of the internet who have all operated and constructed a lot of forums um that measures in millions of users so by using this course we're pretty sure that it will scale uh but um the regulation room people have to reinvent a lot of it themselves okay so when you say the moderators are they are they from got zero community to do moderate this course uh huh yeah so so the discourse provides moderation tools uh and uh v taiwan community is the moderators uh and uh for every single comment there's a huge number of tools that could be used um you can flag it you can put it a batch you can um you can edit part of it there is a moderation history um there's there's many tools a moderator can use you can also give out badges to reward um useful behavior you can also assign um moderation rights to people who are active in the community um the people who freshly registered they cannot post pictures uh and only after they become a good citizen and participate for a while uh do they gain the rights to post pictures um there's there's like thousands of very small things like this in this course that generally encourages civil behavior okay well that that's that's really fascinating i'm going to follow up with the discourse which i haven't been doing much homework on this but i will look at uh a little bit more right as you can see there is like a huge number of knobs uh you can you can tweak uh to basically uh let a system trust the user to a degree where they can post new topics where they can reply where they can post pictures and and and so on and so forth yeah this is very very interesting okay yeah i'm going to tell Angela Angela's about it i think he might find this very interesting okay i'm sorry sorry for you know i asked you to spend more time but thank you very much and perhaps i i probably have to ask you some more questions but in the future and i will let you know no it's just fine those are very good questions and i'm happy that you agree for me to post this online so that the wider research community can benefit from it i'll see oh yeah of course yeah thank you okay cheers take care bye