 So, I have one actually for Ann. I was interested, the use of the word story, do you think that calling feedback a story meant that people were more willing to sort of share their experience rather than something like more like boring like response or feedback? Yeah, that's the term used by care opinion themselves and I suppose it's meant to reflect the feeling that it's their experience. This is how I perceived it because within a care setting obviously it's a lot of, it's a very unequal power relationship and we're kind of used to listening to doctors and listening to the care professionals and I think it's all part of valuing the patient or the citizen's story or their perspective. I think that's what they were coming from. It seems to have worked anyway. Thanks. Luca Ciappuzzi from Paraguay. Thank you very much for the presentation. Very interesting the first presentation about the motivation of people for being involved and I would like to make a question to Christian, based on your analysis or at least on your perception, which was or which is the main motivation for participants that are involved in participatory budgeting processes? Do you mean from the citizen's perspective or from a city that decides to undertake participatory budgeting or both? Mainly for people that are involved in presenting proposals for this. Okay, so it varies a lot and the way I view it is there's usually one technology openness champion within its small bureaucracy that starts lobbying internally for them to adopt this and then what's most interesting that after a couple of iterations and other bureaucrats start to kind of lose fear of engaging with the public, that's when it really starts catching on and a lot of what happens to is all these special in cities, these networks are very tight knit and they talk a lot to each other so at conference similar to this except at the city level they'll talk about an exercise that they did and how happy they were from it so a lot of the expansion of the tool was word of mouth. Hi Christian, so I have a question. Many of the participatory budgeting processes going on most places today are still very much in person, a sort of physical sort of exercise. Do you have any experience of how shifting that process to let's say perhaps any platform or doing electronically has either hiked the numbers of participants or what sort of relationship have you found in places where there used to be a physical, essential sort of nature to the PB to when it has moved to an online sort of platform? Sure, so we found that, don't quote me exactly on this but about there's a four percent like a four time increase because most of the cities we work with they already had town meetings that were open to the government to people where citizens could go participate in person and have similar input on the budget that they do in the online form. The numbers are not as massive as other places because granted these are small places but I think something to note too on that area is that it's a Canada is a very big place where cities are very far from each other and people tend to live outside urban areas a few urban areas so I think having the online portion really helps connect people that would be engaged anyways but that might not want to drive an hour into town to participate. Thank you for your question everyone. Great, thank you all for your presentations really interesting and Cecilia I had a question for you. I was wondering what you think the biggest opportunity is now in Italy both at the federal level and the local level for continuing the transparency and openness work. Before doing this job I worked for a central public administration and now for local. The approach is completely different because when you work for local administration you feel immediately the result on citizens and so this kind of tools online and offline are really used and people you feel that people is really involved because each decision has an impact directly on the day by day life. At central level so at central public administration the transparency is still important but as in indirect impact on citizens so it's important to open data to put in place each process to be clear with people but for me it's very important to invest first of all at local level because each decision the week after has an impact tangible and directly with people. So in my opinion the first approach is to open as much as that as possible at each level but the investment is on local. Hi, Alex Banford I work building government services. The question I want to start with leads on a little bit from the Roman example of having explainers for the digital technology to sort of draw people in. Which worries me because there's quite a useful design principle of thinking that no user should have to understand government services in order to interact with it. You shouldn't have to understand how government works in order to interact with it. If it's been designed in such a way that you need somebody to kind of hold your hand to use it that suggests that there's a lack of a feedback loop between the design of the services and the usability testing that presumably goes on to see if those services are working and I don't know if I've just misunderstood this but I wanted to maybe ask the whole panel have you seen a situation in which government or healthcare have changed their services in response to the feedback that you've been talking about in each of your presentations because I think a lot of the focus has been on about how the citizen or user feels a little bit better after an interaction but not necessarily that government responds in an iterative loop and we're talking about the impact of civic technology, what's the impact? In Rome there was a particular situation, this new portal we released two months ago came, comes after 15 years that nobody changed the website so the previous website was let me say administrative center so it was a picture of the administration. Now we designed this new portal, not only citizen oriented but also service oriented so we re-engineered each process to give services to people and then we not only restyled but redesigned the website and for us it was really important to collect comments from people because nobody was a customer to the new vision and in Rome we had really few people that for example downloaded online certification and so on and so the interaction with the administration was difficult. When we designed the new website we wanted to facilitate each interaction with the administration. Rome is a really huge city also at the level and also the transportation and the movement are difficult and so to have the possibility to download certificate to get online interaction is very important for people so that's why we collected feedback and react on it also to inform citizens about the new vision for more than for a thousand comments and feedback we had so it's not a huge number but it's not a few people. Right so for impact that was one of the things that I wanted to portray was that policy because the way we're in direct democracy and policy makers are influenced by a variety of actors so having or pursuing the one tool in particular will have a policy outcome might be missing a lot of other influences in very complex systems so what's the impact of our tools? Well it has to be viewed within a larger context of influencers and same with as there are other advocacy groups that might push for a different outcome so looking at things like exemplified reports and municipal officials or government officials at least considering feedback because there are a lot of impact more than feedback A on fixing the pool led to outcome A fixing the pool because there might be a lot of other influence in factors such as technical difficulties or a shift in economic in the economics of a city. I hope that answers your question. Hi I think one of the key elements of the care opinion platform is concentrating on learning and change and the flags that I put up each story is flagged according to what happens whether it's got a response or not and whether it leads to a change which is sort of if you like the first level that you start with but after that you see they have quite a bit of social media and blogs associated with different NHS as well and one of the stories that I liked in particular was they described how they were getting quite a bit of feedback in one particular hospital area from children who have I think cystic fibrosis that every time they were sick they had to go into hospital and so they were travelling into hospital they were using up beds but the children's lives were being disrupted as well so obviously they received enough feedback from this so they changed the system of care to nurses going out into the community and delivering their antibiotics in their own homes so they didn't have to be separated from their family and it was a lot less disruptive and that change was led by information from care and there are many examples of that on it sometimes there probably aren't that many changes when you consider the huge numbers of people who participate on it but sometimes it's positive anyway and sometimes it's maybe an amalgamation of things that lead to a change later on down the road is that your story doesn't stop giving after you've submitted it or even gotten a response it's maybe over a period of six months or a year when because it's the people who are in management are able to view these reports and look through them as well and it can lead to further changes then too. Any further questions? Hi my name is Saki Kumagai, World Bank, thank you I think the panel for your presentations. I have a question to Christian, measuring trust in institutions, trusting government is extremely difficult as you mentioned. We have a research component on that and we are encountering similar experiences. Since you mentioned of measuring trust in the context of participatory budgeting, I was wondering if you've considered measuring trust using proxies such as tax, I'm curious to see your views. Measuring the taxi you said? Yeah so it could be, I mean tax is a broad area but one of the more prominent proxies to measure trust or fluctuation in trust is basically looking at the tax data. It could be tax exemption, it could be level of tax collection. Have you thought about it? That would be really interesting. I don't think in the Canadian context it will work as well but somewhere say like in Colombia or even Paraguay where taxes are not collected and then seeing if after implementing one of these tools would be, and I'm just pretty much just sharing my thoughts I'm getting. It would be very interesting to see if there's an increase in the taxation base but again I don't think that will work in the Canadian context but definitely somewhere in transition in post-conflict democracies. Very interesting thank you, thank you for that. Okay there's no further questions, I think we'll wrap up there. Thank you everybody for coming, thank you to our presenters. And as I said before if you're feeling brave and want to join in the photo in about ten minutes time in the foyer that'd be brilliant. If you don't want to and you want to hide I'll probably be hiding with you. So that's fine too, thank you very much.