 Welcome everyone. Thanks all for coming along. I think there are still some people filtering in, but hopefully people will arrive within the next couple of seconds and we can get started. In the meantime, I just wanted to say hello and welcome. Thank you so much for joining us. My name is Rebecca Rumble. Many of you I will have met at various tech techs in the past. But if you don't know me, I'm currently the director of research for my society. And we are the creators of the tech tech program which looks into the impacts of civic technology. Normally pre COVID times in an in person conference, but given given the times we live in we are migrating a lot of our activity online for the moment. So just to let you all know housekeeping. Today's surgery is being recorded. We are taking minutes, and we're going to publish those minutes on the website, the tech tech website which hopefully you've all visited already. Please do if you want to tweet the hashtag tick tech is is our hashtag of choice. We understand that after 18 months of doing absolutely everything online that you might be a bit camera weary. So if you do want to turn your camera off at any point that's fine. We won't be offended. I will keep mine on while we're talking so that you can see me. But yeah, no obligation and no judgment if you're not feeling camera ready today. No problem at all. Just to let you know because we've got a few different platforms that that we're using today for this surgery the zoom chat that the chat function within zoom. That's just for informal chat so obviously you know feel free to interact in there. But we're not going to be monitoring that chat input into the questions or into the kind of main body of the work that we're doing. All of that's going to be done in Padlet, which the link which you should have. Gemma should have sent that to you. So, again, feel free, you know, general to chat in the zoom chat. But if you want to to ask questions or if you want to raise kind of substantial points for discussion, please do that in the Padlet as we go along. So that's our housekeeping. Just to let you know a bit about the tech tech program before before we start off. As I said, it used to just be one annual in person events where we'd all get together. And we'd look at impacts and look at projects and look at research and it was super fun. But I'm not sure about the rest of you here but personally and within civil society, a few people I've been speaking to. We really miss not just going to the big conferences and seeing people and going to the sessions. One of the things that we feel like we've really lost are those wonderful, random conversations that you have with people during the coffee breaks. The really kind of interesting chats that you have between when the sessions are going on where you can either meet people or speak to people that you already know. You can kind of maintain those relationships but you can kind of just sort of say well what's going on in your world, you know, how is it going, what are the big things that are challenging you, what's worrying you, what's been a big win recently, you know, these kind of rather informal, but very informative interactions that you just, you simply don't get at an online conference or an online seminar. Last we've tried to keep the Tic-Tac tradition alive by doing kind of short seminars and short presentations, you know, you lose an awful lot of meaningful interaction if you're just sitting in front of a computer screen listening to someone. Those wonderful conversations that happen during the coffee breaks or when you go for a beer afterwards, you know, a lot of the time those are where the really great, really great relationships begin. We've developed the Tic-Tac Labs program to try and recapture a little bit of that. We realized that nothing will ever come close to being able to sit down with people face to face and have an organic conversation. But we are trying in some small way to recapture a little bit of that over the next kind of year to 18 months where I suspect travel is still going to be a little bit difficult, especially for the kind of global audience and the global network that we have interested in this. So the concept of the program, you know, the concept we're following, we're going to have these regular civic tech surgeries. That's what this is. Welcome to the first one. Apologies if it looks a little nascent or not quite as polished as it should be. This is the first one. So we're very much learning. So the surgeries are about exploring and anything. Some of the issues in civic tech so that we can start figuring out how to address them and, you know, lots of new stuff. I'm sure has come up over the last year and a half as well that was that's completely new stuff. They're all completely new takes on old issues. There's so very much that's happened. I imagine that lots of people have experienced lots of new things and would love to kind of discuss them. The topic of today is public and private collaborations and partnerships, how small civic tech organizations like my society like like many of you here today can start or maximize their collaboration with public and private organizations because those being able to partner with those kinds of organizations often enables you to kind of amplify your impact. But it's not always easier. It's not always obvious how to do that. Well, how to do it meaningfully, how to do it impactfully without compromising, you know, your organization or without running yourself into the ground. So there's lots, there's lots, I think, to kind of share lots of knowledge to to pass around on that subject are themes for all of these civic tech surgeries have been put together by our program steering group. We have a wonderful steering group. The information is on our tech website on each of the steering group members. But yeah, we basically had a did a little bit of research, we had a lot of conversations about kind of identifying big themes that I think are of interest, and that hopefully are important to do the rest of the community. So that's how we've landed on this today there will be other civic tech surgeries over the coming months that look at some other areas. So we have basically in this session, we're going to have a little bit of discussion. We've invited some some researchers and some practitioners along so that they can kind of get the juices flowing they're going to start off having a very short discussion. We're not doing presentations so it's not like sit there listen and then go away, you know, these these introductory thoughts are very much supposed to kind of get things moving start hopefully sparking ideas and discussions that we can then take into the market and hopefully capture lots and lots of really interesting things that are concerning you that you that are of interest to you. And that hopefully we will then take away to workshops to actually distill down a bit to crystallize to figure out okay. So these are things that if we if we can commission this kind of research or if we can develop some learning materials on these things, this will be a benefit to the community. So, as I said, those those workshops will happen afterwards so they'll be a little bit more precise based on all of the information we get here today. They will be a lot smaller groups probably well definitely fewer than 10 people. Anyone here anyone not here but who wanted to be here anyone interested can apply to be part of those specific working groups, if they are really interested in digging more into this and maybe in kind of taking up some of the work that we hope to do as part of the program so who is in engineering everything at the moment behind the scenes while I'm rambling on Gemma will provide all of the details for how to apply for those and timings and everything as we go through. So, that's enough of me talking. I would, I'm so happy we've got some amazing discussions with us today. We've got Alina from Citizen Lab where she is head of government relations and also the co founder so very welcome to you Alina. We have ebtaz Khan from code for Pakistan. He is the government information leads. Thank you very much for joining us. We have Gabriella Rizzano, who is director and co founder of open up in South Africa. Hi, Gabby. Right. And last but not least we have Amanda Clark, who is associate professor at Carlton University. Welcome Amanda. So thank you all very, very much for joining us. I'm just going to fire off some questions and let you guys start the discussion. So, let's kick off. Let's kick off. I mean this one, maybe aimed at Eileen abdahaj and Gabby. So what dilemmas have you faced or are you currently facing when working on public or private civic tech collaborations. Thank you. You're all being very polite aren't you and expecting someone else to go first. Okay, let's let's go. And then Gabby is that okay. If you could just give about maybe three minutes each that'd be lovely. Yeah. Yeah, I think we were. I mean no one wanted to go first. So yeah my name is Alina one of the co founders that at Citizen Lab, and we've built a civic engagements platform that is mainly used by by local governments to engage their constituents and citizens on many different topics. So we now work in, I think more than 15 countries we started in in Belgium, but we're active in different European countries but also more and more in the Americas, both Latin and and North America. I think we've, we've done this now for over the over the past six years. And I think not many dilemmas or challenges have have changed I think they're still quite the same than than when we got started. So I'll just list a few right that I've, I've, I have a bullet list of a few items. So I guess first of all, I saw also that someone introduced this in the chat. I think like public private partnerships, or collaborations you can so interpret them in in different ways but I guess like we often really work in like the contractor type of collaboration right will be a supplier of of software we work with the governments as they are our clients so I guess one of the first dilemmas or challenges is always procurement how do you get in how do you get this collaboration started. How do you make sure that I mean you get paid for for what you're doing. But of course procurement can be quite of a long and difficult process I can think we can further go into that later. Then when one of the dilemmas we're also facing is like, we want to get to value quick and impact quick like I also work on impact that citizen lab so how can we make sure that our platform is used in in a good way that we get to impact fast Then we kind of easily face very silo structured as we all know governments. We work with local governments but even there, although they can sometimes be pretty small like midsize cities of 50,000 inhabitants, you'll still have like those very silo structured organizations, no matter which topic right we work a lot on urban planning mobility climate. So you kind of always come in within one department whereas you actually want to have impact on the whole organization. And so that's often a dilemma right like how much do you invest in like a good internal organization making sure that they take the lead on things, but not spending like months and years before getting to to any, any results. And then last is like my third point could be just a general understanding of software and software as a as a service. We all work in in technology. And there's more and more interest in technology I guess definitely also in the past year and months people have been there has been a huge push in digitization but still in our I mean we are software as a service. We still kind of expect a lot of, yeah, customizations there's not too much understanding of how software is often built so what can be easily done what cannot be done. So that's also sometimes difficult because like our key contact people don't really understand how to adjust things how it's built, how we are facing decisions. So I think like the three main dilemmas. Okay. So yeah I think I just briefly mentioned what corporate Pakistan is and then come to the dilemmas so corporate Pakistan for all of all of you who do not know what corporate Pakistan is. We are not not for profit and we've basically work on bridging the gap between citizens and the government in Pakistan through technology. And we've been working with various public and private sector organizations over the past seven years. And finally with the government we mostly help them improve their public services as well as this, their business processes through our fellowship program and on the other hand we try to empower educate and mobilize the community through our hackathons and civic innovation labs. When, as I mentioned we've been working with the public and private sector over the past seven years now and again, it's not easy to work in a public and private sector setting. And obviously all of it comes with its own set of challenges and we recently did publish. I mean a couple of years ago we did publish a research paper on the learnings and the outcomes that we've had so far from from the fellowship program today I'll just briefly talk about just a few or the major ones that I can think of. Right now so I think the first and the, the foremost challenge that we faced is, is to build that understanding around civic tech and making people understand making the public sector as well as the private sector understand what civic tech is. Whenever you go out there you meet someone, you pitch to them what you do and they come back to you oh okay so you are, you develop softwares don't you. And that's that's the sort of feedback that you get from from from the public and private sector so. And I don't blame them for that because I think they are used to such a setting where they generally interact with such services, such people who, who think the only value, only value that technology adds is through developing software solutions through websites or through mobile applications and they do not think out of the box on how to add value by co-creating solutions. Then the, the second challenge that we basically recently I think we recently identified is but something that we've been working on for the past seven years and but is that we generally as a whole as a nation or as a community in the public as well as the private sector we do not understand what our problems are to the core. So for example, we recently conducted a gov tech innovation challenge where we crowdsource problem statements from the from the public, just asking them to list down what the biggest problems they're facing. So they did get some very interesting problem statement from the public but then most of them were very basic so one of the problem statements if I can say was about that there is noise pollution and that there's a noise in a certain area of the city, right? But then if you dissect it further you'll notice that that I mean that from from a public's perspective that perspective from from organizations that might be a problem but when you dissect it you, you should be able to list down why is the noise, why is the noise pollution there, right? So it can be due to many reasons. It can be due to let's say there is excessive traffic there, there is urbanization, there is increase in population, there are factories there so there can be multiple reasons and then when you dissect it further then you come to know so for example if the traffic is the major cause of causing noise pollution then how or why is the traffic there? So is it because there is lack of traffic infrastructure or if there is lack of traffic HR, traffic ordinance, is it because the lanes, the traffic lanes, they are narrower. So I think in general we still need to understand what our problems are and that's one of the challenges that we face, we've been trying to work on that but this is something that we recently have started working on. And then again something that Alina also spoke about is building trust within the government. So again when you go to them, you pitch to them, the questions start coming in, why are you here, why are you trying to help us? And again when we talk about then accountability and increasing efficiency then that is a big question mark because they come back to us by saying why are you doing that? Why do you want our data to be published, right? There are other questions like that and these are some of the challenges that we faced and the last one that I'll briefly talk about is the adoption of civic tech solutions over time, adoption maintenance basically. Whenever you or a CSO or the organization's work on a certain product or service and they offer it to the community to whoever the stakeholders are. So ensuring that that solution is being used, I think that is a huge challenge. So developing technology is not the hardest part here but ensuring that the solution is sustained over time is a major barrier here. And just to again give you a small example, we've worked on developing an open data portal here in Pakistan for a government agency and it has been there for the past two years. We've uploaded a couple of thousands of data sets to the portal but it has not been launched yet just because of bureaucratic hurdles they just want policies to be in place before launching a technology. So these are some of the challenges that I can think of on top of my mind right now. Thanks so much. Kaby, what are your thoughts? I have one down. So just to also flag in South Africa we're having load shedding so I might randomly disconnect at full but I'll come back on some of the exciting experiences we have here. Hi, I am the director of open app. We're a civic tech lab based in Cape Town that we've been around since about 2013. And we work on using data technology and innovation methods actually sort of just data and technology but innovation methods for activating citizens for positive social change, largely in a South African context. So I'm involved in research and law and policy at a regional level. So I'll, I can talk to two different areas, one of them is public private partnership partnerships in the context of our experiences and undertaking them. The other is looking more at public private partnerships at a policy and the national level and some of the challenges, particularly for South Africa. From that perspective, I'll do it really quickly though. We've done civic tech projects at both national and local government level. So that's really interesting. But, you know, you see the capacity challenges running through all of the spheres of government, to be honest. But, you know, I think to maybe talk more to another side of things, there's largely, there's often a conflict, a substantive conflict and procedural conflict or methodological conflict that arises when you're trying to undertake public private partnerships. I mean, the basic logical one is, you know, for organizations like ours, which are, you know, very spiritually connected to sort of agile development, you know, and agile methodologies. It's very difficult to implement those projects in contexts which are, you know, entirely waterfall and they just, you know, a lot of entities just don't know how to work that way. And what it does lead to to sort of a substantive conflicts often between, you know, us trying to seek to create a kind of impact and our public partners very fundamentally being focused on, on, you know, the performance of certain deliverables whether or not those achieve impacts. And I'm talking very loosely, I suppose, but you know, we've seen it in a project that we've just been implementing quite torturously over a two month period, where, you know, it's more important than sort of sort of activities are being implemented or deliverables are being produced for the international government that we know aren't going to have an impact with the local government actors that it's meant to capacitate because there's been no attempt to get by and from the local government partners by national government. Something that we raised from the beginning, you know, and obviously very, it's the antithesis of how we see work actually needing to happen, but in a two month period, it's that's been, that's been a sacrifice that we've had to just set up and deal with, you know, but we, we take our own lessons and we, and we learn how to deal with those. Some of the strategies we use are to try and, you know, we tend to like a longer process so that we can sort of on board and create cultures of sharing between us and our partners. We've seen that work with government quite well like in the launching of the Kamali but I mean that was a project that's banned. So that was national treasury's open budget portal. But you know that's a project that's banned about three years, you know, and so you can afford to invest in building relationships that mean you can work more collaboratively. But that's not always possible. I think it's also you can construct your contracts very well and craftily. And, you know, we always as part of our pitches, we always ensure that we make very clear how we work and what our methodology is within our tendering process. So within our proposals we make it very clear how we produce our work. And in contracting we try and contract to deliverables that are tied to impact and not necessary product although that can be very difficult to negotiate but it's a practical strategy that you can use. And then if you I mean if you turn more broadly to sort of some of the bigger issues in public private partnerships at a local level. I've added in a little case study there I did on some of the digital hegemonies that exist in relation to public private partnerships that we saw expressed in sort of covert 19 related surveillance technologies in South Africa that can be a little instructive. Because the reality is particularly in South Africa but you do see it regionally that the in house capacities or the public sector capacities for both using agile methodologies but also implementing technologies are necessarily there or that I don't like to be the person who says that in a condescending way it's just a practical reality of what we've seen and a lot of that is not a prioritization of technical skills within public service departments. Which is, you know, something that needs to happen in order for these kinds of dependencies or codependencies that exist between government and private sector partners, existing and resulting in terrible like very negative things like which you see with a lot of technology projects. And you know the repetition or the repeated use of partners for the delivery of products who aren't focused on public service or, you know, public outcomes. And I mean you saw that have devastating results in South African, in the South African national context when the outsourcing of social grants, the distribution of social grants was outsourced to private sector partner and the conflict that arose there. A lot of that relates to procurement and a lot of that, you know, I think for us as civic tech advocates and I mean certainly from our perspective because we work so much in transparency, you know, there, you know, there are long term advocacy interventions that need to happen in procurement and transparency, which are necessary not just for the greater good, you know, like and transparency is a good but also for implementing more successful social innovation. So, that's my touch. Thank you very much all of you really thoughtful and detailed discussions there. I'll just keep the ball rolling but keep it hopefully rolling towards a very positive destination. I'm just curious as to what you think might help you address some of those issues I mean obviously you all talked about like a whole spectrum of things there so I'm not expecting a unified theory of how to fix everything. What are there are there small fixes are there are there small things or are there actually gaps that you just think look we don't even know enough about the problem we know there's a problem and we know a bit about the problem. One of the one of the themes running through what quite a lot of you are saying is that a lot of the problems at our specific end of things are kind of the result of institutional or other kinds of issues that are kind of inside some of these organizations or governments. So, are there any things that you think either a that you you have been doing when you have identified that that could be solutions, or be other areas where you think, well, I really wish someone would go away. Like, and do some research into this so that we actually understand it better, so we can, so we can look at about it you know their resource gaps there anywhere. So trying to go back the other way this time go Gabby have to house in a lean. Keeping it exciting, we bring it around I was just playing with my hair. And so I mentioned some of the, you know, practical things we do like with contracting for instance, I think another thing, which is like, we found as a strategy which is like a gentle strategy for bringing people in to space which can focus a lot more on impact, then, then just deliverables although you know I know also a lot of those that focus on water for men deliverables has to do with how government budgets work. And there's not a lot you can do around that, you know, you know, fiscal dumping you just want to be on the positive end of fiscal dumping. Which sounds hilarious when I said, and so I think, you know, another thing that we like strategy we found quite useful is to always make sure we put within budget space for user centered research, and to really emphasize user centered research in the creation of our products, because we find it's often a gentler way of steering people towards, you know, higher impact is not to say, you know, open up wants to do the projects this way but is to say, you know, we've spoken to users, and these are users on the eating and I mean we know it's both better from it, you know, an innovation design but we also find it a useful kind of political tool because it kind of de-emphasize it. Yeah, it sort of reduces conflict it doesn't seem like we have competing interests what we instead say we focus on the citizen or user and say that these are what those interests use so we find that a very useful strategy I think there's a little person you just like an evil habit. So, yeah, that would be some practical advice and I think works for us. Thanks Gaby, really useful. Yes, thanks. So yeah, I mean, it's somewhat similar to what Gaby just mentioned and just going to her point earlier so since we've been working here for the past seven years now and we've had our own share of learnings as well and we've tried to adopt the challenges that we face so I mean I earlier spoke about the government officials the private and public institutions not entirely understanding the concept of civic tech. And through now through our fellowships that is one of the important components that we've identified and it's a part of the curriculum there now where we not only train the fellows on what civic tech is and how the process works. We also try to encourage the public office holders to be a part of those boot camps be a part of those sessions so that they can also learn how all of this works. So just trying to instill the spirit of civic tech throughout the public sector. I mean, we cannot do that entirely but we are trying in bits and pieces to to educate some of them to the extent that we can. And I mean the end goal is trying to empower the staff so that they can instead of waiting for a following someone they can go and be those change makers, rather than following someone. And we've we've seen the results already so in one of the provinces that we work a lot in is the heaven for province and now after seven years I can see a clear difference in the understanding of how they used to interpret what civic tech was and now how they interpret how they are now leading the change themselves and they come to us with requests for that we need help with X, Y, Z. And then guy be also mentioned that user research and we again we specifically focus a lot on conducting user research and identifying the problem to the code before taking on anything. And that's again a part of the curriculum that we have the fellowship curriculum that we have and again that is being taught not only to the fellows but also to the government and public sector employees and and and then I mean entirely this this is not only just educating them but to build the capacity within the government and the public sector. And just giving them the confidence to take on challenges that challenges themselves to be to bring that change. And then just giving them some some success stories out of the fellowship program so initially when we started I remember we whenever we used to go and pitch to the government. And gave them examples from likes of code for America from code from from the broader code for all community, I'd say, but now since we've, we have a couple of success stories from within go for Pakistan so that that motivates them. And they are now more into co creating solutions with it with the public with the citizens rather than just working on the problems themselves and building solutions around them. And then one other thing that we've learned along the way is to build with all the stakeholders so again as Gabby mentioned that they also work in a very agile environment but the government is generally not used to working in an agile environment so now we are again trying to make them work in that agile environment if even if they're not inclined towards it so our fellows go there we ensure that our fellows the teams, whoever is working on a certain product they go and they work with them they give them regular updates they follow the release early and release often concept and just give them something so that they can feel the ownership of whatever they want is working on and then lastly something again related to the inspiration is storytelling which we've been focusing on lately is is just to tell is is to make them learn how to tell stories as well as we as go for Pakistan we've also been started doing that. And so yeah I mean I mean that those are just some of the things that we've done we've learned along the way and we are trying to implement them I mean not to the extent that we can but we've started we've we're trying our best to bring the change within the public and private sectors through the work that we do. I love it when anyone trying to get the spirit of civic tech into public sector bodies warms my heart. Alina do you want to do you want to discuss your thoughts as well. Yeah, so I think I, I really follow what what has just been been said over the past 1520 20 minutes I think we see the recurring challenges and I just want to share some of the of the small things. So what we're doing I think what Gabby mentioned and what I've also seen in the chat is like, putting those impact goals into contracts and trying to make that very measurable in a collaboration is I think key unfortunately we haven't seen it much in procurement right I mean they might like at the beginning of a tender there might be the goal but then in the end it's not really measured along the way so what we try to do is to to make it measurable so I just put it in in the chat. I mean we try to also communicate our theory of change to the governments we're working with and we were trying to say like okay, this is the SDG we're working on this is how it boils down to indicators that's how we measure it that's how you are performing this is where to other governments. This is where you can work on. So I guess I mean it's it's pretty difficult but we try to to bring it into into government to start measuring their their progress and have that basically as a steering wheel for for what's important and what's important, then to I guess is like building capacity, same thing. It's really key to connect people with with each other. We often identify like the real ambassadors and like the people within the government that get things done. And it doesn't at all depend on hierarchy right it can be a trainee that comes in or it can be whoever. We try to connect these people to each other, and we've really seen that working if we invest in the community and if we see like okay that person they might not have the skills today but let's connect them with someone who has a similar position in another country. But they're doing pretty well that we can see that investing in them and mainly connecting them with each other really helps in in building that that capacity so I guess that community building is really key it's great also to have many people here today but it would be interesting to also have more people from from government so that they participate and that they also learn about our challenges and it's very transparent that we can have that space and not just like the more contractual relationship. And then I guess, last but not least is also like, I think, Abitaj also mentioned it, it's like, how do you make sure that there's sustainability in the, in the initiatives that that you're launching how do you go beyond just pilots I think pilots are important to test things and to be innovative, but then you do need to have that long term commitment so. And that that's often a dilemma right like you need to be able to be flexible during the contract but still have a long term collaboration to have that sustainability. And that's what we've been been doing from the beginning like we really say like okay we tried to say no as much as possible now to short term contract because it's just like we know that there's just such a risk that will do work that then is not being continued because of often also changes in political leadership elections right or often what like we work with with local government someone comes in we've been doing great work over a few years and then I like okay let's change everything now. So that can be really frustrating. An awful lot of people on this call will say those frustrations and those experiences. Thank you. Thank you all so much for that. Really, really interesting discussion so far. I'm conscious that you know people have been listening people have been chatting a little bit informally on the zoom. We have our full padlet online Gemma has there she goes she has just shared the link again in the chat. We have so many people here with so much experience so many different interests, lots of experimentation around these themes under their belts. It would be fantastic if we could maybe take about five minutes right now. One of our speakers a little bit of a break in case anyone needs a very quick comfort break that's fine. But yeah if we take about five minutes now it'd be great if everyone could jump on the padlet. There are these questions that we've been discussing on that. As I said earlier, everything that we can get from you guys now in terms of ideas in terms of identifying issues in terms of identifying things that you need as civic tech organizations that would help you go in your work going forward. Throw all those things under the right headings on the padlet we'd be really grateful. And it will, as I say it will inform a lot of the other work that the project does from now on. So it is. I'm 47 where I am I know we have a truly global audience here so I'm not exactly sure where it is where you are what it is where you are. But if we if you take five minutes now, and we'll come back and continue the conversation. Thanks everyone to you in five. And that's your five minutes everyone. We have the excellent timer here to keep us on track. Thank you Gemma. And thank you all for your contributions to the padlet as they say we really appreciate all of your thoughts. It's going to be incredibly useful going forward so yeah excellent and the padlet is open as we go so please you know if you have other thoughts if you if you have other things that you want to contribute while people are speaking if something just clicks you know like Bob goes off please throw it in there. It's all going to be it's all going to be massively useful for us. We are very lucky that Gabby is wearing two hats today as a practitioner and as a researcher she's produced some fabulous policy and other research items that are available online Gabby feel free to share them around. We also have Amanda Clark joining us. He was a professor at Carleton University Amanda has done a lot of research around these kinds of themes so it's going to be great to have her thoughts as well. So, I suppose you know that loads of themes and loads of things have already come out here today. I suppose can we maybe just like reflect a little bit we've had you know we've been going for about an hour now so Amanda maybe do you want to kind of jump in give us your thoughts, especially around things like I've noticed on the padlet. No one's putting any, oh there's definitely a great piece of research about this that we can all learn from. So maybe you could enlighten us or talk maybe more broadly about the kind of these kinds of themes that come up in your work. I have to say I'm like thoroughly enjoying the conversation today I've really missed the tech tech community so this is great thanks for bringing us all together. And it's so nice as a researcher to have the opportunity to learn from those working in the field and it's sort of like it feels like I'm doing my research while I'm just sitting here just by curiously listening to you so I was quite captivated by everyone's questions. I think in terms of the existing evidence based. One of the things is that history is repeating itself in many ways. In the area of public administration research that looks at co production procurement public private partnerships which is a pretty long standing field of research and I think really a lot of that work started getting going in the 70s and 80s. So as governments started to look more to kind of external actors to partner with for service delivery or for policy design. Researcher started uncovering a lot of the same issues that we're hearing about today so issues of you know government contracting requirements being too onerous. You sort of accountability regimes and government being difficult to square with the reality of limited resources and some of the nonprofit sectors nonprofit sector organizations. And sort of the strict hierarchies and silos of government making it difficult to find access points and to sort of more organically develop relationships. And especially I think the point around kind of shifting governments not just changes in government making it difficult to sustain projects and build relationships but also just internal movement of civil servants, making it, you know, the person you used to connect with on a certain issue is suddenly no longer suddenly no longer working in that role and it's like there's this gap and like who do I contact now right so a lot of history repeating itself. I think the question is. What can we draw from that research to improve practice today and part of, I think what might be a little bit depressing about that is whenever history repeats itself it suggests we haven't learned any lessons right so maybe what we need is to take a new approach to some of this research. As I was listening, one of the things that I think is really exciting about the opportunity right now is especially emerging from the pandemic where a lot of governments were forced to reckon with on the one hand the absolute essential and sort of central role of digital technologies in our lives, which has you know predated the pandemic but it was easy for governments to ignore. And then the second was that a lot of governments were forced to reckon with their lack of digital capacity right like whether it was inability to do contact tracing or to set up vaccine scheduling services on, or even to be effective public communicators and kind of digital platforms. So all that to say there's this inflection point where it could be a great opportunity to reinvigorate the role of civic tech in government, and really strengthen those relationships in a way that avoids other pitfalls that are three opening speakers really like eloquently laid out for us. What I think would be really helpful in terms of research to drive this policy work would be very vivid case studies that illustrate to us sort of these kinds of partnerships can work very well. And then also when they're not working well why like what needs to change in government in order to facilitate. And I think that if they were written in an accessible way, and had were very helpful examples I actually think you could capture the political imagination in a way that you know there is this. There's clearly an appetite I think amongst political leadership to think creatively about the opportunity and to improve services and there's all whether you're on the right or the left there's excitement about working with outside partners for a whole range of reasons so you know I think that we've kind of got this. There's like a group of bureaucrats you really get this and I think all of us can think of those like digit whether it's digital service teams or the people working in open data or those who were former civic tech kind of outside government they kind of get it now they need to get their political leaders on board to deal with some of the larger structural changes that you've outlined so one like space where I think some, you know really interesting case studies could be done would be showing how how a service that has never worked can suddenly work. If you create space for outside partners to work in more agile ways with government. You have a lot of case studies that show that and really kind of clearly accessible ways that maybe you could put across the desk with minister and say like when your digital team those like little up starts in the corner that you hired five of who have tech talent. When they tell you that you need to change our budgeting processes because you undercut scope for user research because you set out deliverables early on in the process like this is what we mean this is an example of where it worked really well and here's some like solid evidence of the impact on citizens lives and you know that could be I think that would be really great so I think it's incumbent on researchers to try to generate more of those case studies and write them up. And I have ideas for other research projects that that like as I was listening I was just scribbling down lots of things we could do but I'm also conscious that other people need to speak so I'm going to hold off for now but I'd love to come back to some of those other ideas as well but that's my. Thanks so much Amanda that very, very wonderful distillation actually of some of the things that have been said here today and I'm definitely going to come back to a couple of those points and maybe get some other, some other views. Gabby, I don't know if you would like to kind of reflect a little bit on on the things people have been saying here and anything. Yeah, you think might be useful in future or a good way forward or you know opportunities that have arisen in the last few years. In research in particular, I'm guessing, but you know I think, I mean, not less of all because it's part of our sort of, you know, our politic what we view as political opportunity in South Africa is the ability to look at, you know, really deep dive into local government. You know, really discreet community case studies and not with a pure vision of scaling up which is always that terminology that gets used and abused quite, you know, by funders in particular. I'm actually thinking, starting to develop a methodology, which is a little bit more honest about what the factors need to be considered in scaling up in terms of context that there are actual methodologies or you know, that could could exist which would help you. Possibly on some level calculate, you know, opportunities from different civic tech projects, what needs to be looked up or if they wanted to be scaled into different contexts and those kinds of things, which I think is an immense opportunity, particularly, you know, in the region within our region there are a lot of projects that happen very, which tend to be limited to urban highly connected environments.