 Oh, that was a lot louder than I expected. Hi, everyone. My name is Christian Medina Ramirez. Thank you for having me here at TicTac. And I'm going to use this time to talk about a research project we undertook at Open North, the organization I work for, on our online participatory budgeting platform. And before I dive into that, I'll talk a little bit about ourselves. As I mentioned, we're a Montreal-based organization. We are guided by doing applied research, a lot of multi-stakeholder engagement partnerships. And we try to boost the effectiveness of the community and social movements and initiatives. And I know it's a little rude to talk about ourselves. But one of the purposes of this presentation is pushing the idea that we are civic technology, open data people that are going into the participatory budgeting space. And that's something that's not happening enough, and in my humble opinion. That's some of the people who we work with, if one of you is here. Thank you for working with us. All right, so we can dive into the presentation. In this 20 minutes or so, I will demonstrate how our light approach to participatory budgeting has had actual impact on the Canadian context and impact, as we've learned over the past two days, is a very loaded word. So I'm going to begin by discussing our approach to measure and impact, which might be a little different from what we traditionally see in the participatory budgeting field. I will also, given our definition of impact, try to situate this within the larger civic technology and impact literature. And then I'll talk about the exercise, some of the results we had. And then I'll keep pushing some of our research agenda on all of you. So again, so what was the motivation for research? So we found that we were sitting on five years of fairly consistent data on participatory budgeting. It was a longitudinal, relatively large, and study. And this don't tend to exist very much in participatory budgeting. The way the field developed was very suddenly and pushed by a variety of different actors. And it sort of developed independent of each other, which is very, very different from the civic technology and open data movements, which have more centralized events. So participatory budgeting became widely popular really fast, but we never really understood what impact it was having. And this, again, reminds me of a lot of the conversations we've been having this past couple of days in the unexpected results of some of the work we have undertaken. And we're also thinking of starting to bring this tool abroad. Citizen budget, our tool, became widely popular in Canada, somewhat unexpectedly. It is used now by over 100 cities, and it's been used close to 150 times, all across Canada and a few in the states and France as well. So what does our tool look like? It looks something like this. We use real physical data provided to us by the municipalities. We work very closely with the bureaucrat on the data. And then we adapt this data so that citizens can easily understand it and use it and play around with it. One of the main purposes of the tool is educational, showing citizens the drawbacks and the advantages of choosing different services and of shifting the city budget. And in this way, hopefully, lead to better communications and better understanding of government functions. So when we already took this research, we wanted to ask, what do we mean by impact? Is it just brute measures, such as number of instances of participation or web visits or bounce rates, things we've observed being tracked in other civic technology tools? We also wonder if we should pin impact on policy outcome, that's a citizen's particular preference or feedback lead to a particular policy outcome. And we also wonder if it was citizen perceptions of government. And I'm going to put a little asterisk on that because measuring trust and trusting government, it's quite tricky. And as is very often the case in this kind of research, we found that it was a combination of all this. So we came up with this, well, we didn't came up with, but we found this framework in the literature that we found could be really useful for tracking this impact. And we were trying to move beyond our traditional understanding of what technology tools should look and what we should track. So again, we came with this dichotomy of tangible and intangible impacts. And intangible is a bit of a misnomer because these are very trackable. So under tangible, and I'll give some examples from our research on what this mean, we have very physical things, reports, policies, policy decisions. And intangible are more the bread and butter of our field, participant empowerment, increased trust, and all these very buzzwordy words we sometimes are guilty of using. Okay, so what did this sort of impact, what were we exactly looking for in our 90 plus cities database? Well, it's things like Councilor Linda Boosie in the city of Yellowknife in the Canadian North using a citizen budget report and report to justify one of her votes in front of council. Or it was the deputy major of Morenville, a small town in Alberta, deciding to, under a unanimous motion, defer a vote on the public city library because after consulting with citizens, they found that citizens did not want the library to be defunded. Or finally, the town of Perry Sound, which is a small rural community of 5,000 people in Ontario, my home province sort of, where they decided to integrate participatory budgeting schemes into their overall budgetary processes. So every year before doing the budget, they decided to ingrain budgetary engagement as part of that. And if we go back to our little outcomes form, we see evidence of policy decisions of policies. Maybe the policy wasn't implemented by the fact that they deferred that decision points to new processes and new institutions. On the intangible side, we had this very nice quote from Robert Nicolay, which is a city manager in Grand Perry, another small city in northern Alberta. And he mentioned how he found that after implementing citizen budget, this online participatory budgeting tool for several years, he found that his citizens were far more receptive and that they were already used to participating in this kind of engagement, that they were contributing very helpful feedback to him that might have not been there present if they had not implemented this sort of tool or this engagement mechanism. And again, these points to all these things that we always look for, participant empowerment, improvements of understanding of government and willingness to participate in the future. And again, I'll take a little bit of a time discussing impact because in the participatory budgeting field and perhaps in the wider inclusive technology field, we had this high expectation at the beginning that we found the panacea that was gonna solve all our democratic problems, that we were gonna finally evolve into this sort of Aristotelian democratic society, that we would reach this perfect democracy. But after 30 years of participatory budgeting in particular, we found that that wasn't the case. And some of the participatory budgeting pioneers, such as Porto Alegre and Belo Horizonte in Brazil, were starting to become very disillusioned with the idea of participatory budgeting. And I'm not here to say that participatory budgeting has had no impact. What I want to say is that it's had impact we weren't tracking and that perhaps as a community, we should start tracking these forms of impact instead of focusing on policy outcomes or on participation levels. And so to further advance this point, we used our large quantitative data set to test a couple of other long held beliefs in the field. So does previous democratic culture affect people uptake in participatory budgeting or online schemes. And it was very interesting, the last presentation in Paraguay, seeing how people were really encouraged in the context to participate, even though they didn't expect their feedback to be heard. And it's very telling of the place where the Paraguayan democracy is compared to other types of democracies. And the second thing we tested is does low participation lead to cities, which was our main measure of analysis, halting use of participatory budgeting tools or online tools. And we found that neither of this was the case that despite having different democratic cultures or democratic healths, and here I want to end the interest of transparency. I want to say that we used levels like previous levels of voting in each city as a proxy for measuring democratic health. And we did this because that's the one consistent measure across cities that we can compare. So one vote once one person, and it doesn't vary that much from city to city. So we figured this could be a good proxy measure. Anyways, we found that there was no real correlation between how healthy a city's democracy was compared to whether they decided to participate a lot or not on our exercise. And we also tracked cities that have done this for four or five years to see if changes in the participation rates led to them not having the same source of impact that I was discussing earlier in the qualitative portion of the exercise. And we found that no, this wasn't the case either. Cities still found, despite having not necessarily high participation numbers or despite having different spikes in participation numbers that they kept participating and they kept getting some of the wonderful benefits of social learning that we observed. So this is all super encouraging for us. So after finding this, and after keep going with the research, we wanted to go a little farther with what we can do with these data sets. And we wanted to go a little bit farther with the participatory budgeting field. So we started asking how can we use this information, this massive amounts of information we capture in this participatory budgeting tools and in this engagement tools we use online with open data, how we can link them so that others outside our tool and outside our community can use them. And then since trust is so hard to measure, we still wanna find out how should we go about measuring trust. So we started collaborating with the Privy Council Office in Canada, which is the office in charge of collating all the ministry information for the Prime Minister to check and to be able to make informed decisions. And we decided to collaborate with them so that in every chance they get to consult and feedback for citizens, that they can actually use that feedback in data-driven solutions and transforming a lot of that qualitative feedback into usable data. And also to advise them in this feedback so that in gathering this feedback so that they themselves could use this feedback to transform it into a qualitative, quantitative, into quantitative information. And then for trust is something we started recently. And how do we measure trust in government? And we found that it's really tricky because perceptions of trust are tied to understandings and misunderstandings of government roles. A recent survey by the Pew Research Center found that only 19% of Americans trusted their government. However, when asked about specific services that the government provided, most of them agreed that government was doing a really good job and except for one, which was in immigration and no, this was before Donald Trump and the Mexican wall became a thing. So the point with this slide and with this point is that that pinning impact of our tools on trust is not necessarily the best measure because government trust is shaped by cultural attitudes and it's shaped by too many different actors such as the media, misinformation. Well, you were at a morning on Twitter and you saw an angry post by Donald Trump. And on that note too, there was a research by a marketing firm that showed that the most trusted institutions in different countries were the Chinese Communist Party and Vladimir Putin, two very anti-democratic entities. Pinning the health of our democracies and the impact of our tools on perceptions of trust might not be the best idea. Again, we saw other things in citizen budget that speak to the wider civic technology literature. There's institutionalization of the tools. The tools survive changes in government and changes in staff. It's slow tech and it doesn't require a lot of expertise which is important for designing these kind of tools and it's easy to implement for the citizens. So it's not and for the city employee as it's not as hard to grasp with the technology. And there's city official buying, there's a co-creation process if you will between us and the bureaucrat. The bureaucrat provides us the data, the bureaucrat decides the side itself and we're there more as a support. And this really leads to, I work a lot with these people and talk with a lot of municipal leaders and this really leads them to be very invested in the tool. And so what are some counter arguments on this presentation that I've given on our approach that it's a top heavy approach of participatory budgeting is traditionally viewed as a bottom up approach or it was at its start. So that's a fair criticism. It is a top heavy approach and has developed into a top heavy approach but we just need to look out for some of the dangers that involve this top heavy approach such as the process being hijacked by political interests or the power dynamics of government simply missing a portion of the population under feedback and this can be mitigated by collaborating with civil society, we hope. And will it translate to other context? Well, we don't know. There's some interesting work coming out of Kenya since they passed some legislation to make participatory budgeting and online consultation with citizens mandatory in budgetary processing and budgetary decision making. So it will be very interesting to see what comes out of Kenya and if there's any possibilities to compare with Canada. And are we cherry picking impact? Are we just looking for things that look like impact where there aren't any? Are we just adapting our definition of impact so we can pat ourselves in the back and say we did a good job? And that's a fair criticism and we did control for a lot of variables and we double blinded the study but that being said, the point of the presentation is not to say that other forms of impact such as tracking instances of participation or border perceptions are completely nullifiable. It's the point to say that there's way more other forms of impact that we should be tracking and by tracking these forms of impact we can come to better understand the effect that our tools are having and give us ourselves a little breathing room. And on that note, I would like to thank you for paying attention to me. Have a good day. So contact me if you have any questions. Thank you. Thank you, Christian.