 Welcome to Inquiring into the Empowerment of Communities, first of all we have impacts of discussion platforms for young people with Miep Lenoir, we're gonna cut questions from these sessions I'm afraid just so we can keep the time and everybody can get to trains and planes and things like that. I'd like a welcome to you to the stage, Miep. Thank you everyone. My name is Miep Lenoir, I hope you had time to recover a bit from the last session. I'm going to talk to you about digital communities for social change, specifically a program called Citizens Voice that we run at R&W media. First of all I'll talk a bit about the approaches that we have to create alternative spaces for civic action after which I'll present a bit the impact that we found on users, knowledge, attitudes and behavior. So yeah we work in restrictive settings which I think by now could be the entire world but these are some of the platforms we support. The first one is Yaga in Burundi, Habari RDC in Congo DRC, Bembere in Mali, Hunalibia in Libya, Manassati 30 in Yemen and justice for her or I think in Chinese it's orange umbrella in China and so we work with young people between 15 and 30 years old and so they are both our end users and our drivers of the local teams who consist mainly out of media specialists, change makers, activists. So apart from Citizens Voice we have two other programs called R&DCs, one is this is a training center for impact journalism and Love Matters is our platforms around love sex and relationships. We work in total in 13 countries and of course in all different contexts and all contextualized programs. So when talking about creating alternative digital civic spaces this is a bit of the potential that the digital media can can offer. Unfortunately these are numbers 2017 but I think in general mostly the difference from 2018 is that the video views have gone up incredibly. You can also see that most of our traffic is through social media although all platforms have websites, most of the traffic that we gather is through Facebook actually and in China it's I think Weibo. I think this is a nice example of scale. We found out last week that in Burundi one of the mobile companies is actually using the Yaga logo, you see it here, as in in their ads. So you see that it's actually bigger than the Facebook logo. I'm not sure what that means. So I'll discuss a bit about our theory of change and then we delve a bit into the approaches and then I'll talk a bit about impact. So our theory of change has two pathways. The one the first one focuses on creating that alternative space that I was talking about. So let's start with at one where we have young people in the countries that we work who live in restrictive settings where freedom of expression is limited, freedom of association is limited, digital or public space limited. So their platforms are created where they can come from all kinds of different backgrounds. So we make sure that people across all kinds of divides in Yemen, it's north-south, in DRC it's more regional. So that we access that people from different ideologies and demographics have access to their platforms. It's also a matter of access to pluralistic and liable trusted information. So that's why they come and visit the platforms. Mostly of course to consume but also to exchange and share opinions and our long-term outcomes that we assume and that we are testing with the research that we don't. It's actually we think that by exchanging, by learning about other viewpoints and about reading maybe about different multiple viewpoints, people are more open to different viewpoints and also in the long term that we would love that leads to more acceptance of diversity as well as challenging restrictive norms and prejudices. So keep that in mind when I present the research because that is actually, yeah, that we want one of our outcomes to be. The second track is citizens feedback because of course thousands of people that visit our platforms, they provide us with loads of information on the needs and the views they have and this information is collected, analyzed, fed with other media and advocacy organizations, change makers, so to influence decision-making. We've had already quite a few examples where this was successful. For example, the Yemen platform was asked to represent Yemeni youth at the Stockholm Peace Talks end of 2018. Yeah, so this is a bit about our approach. I think when yesterday, Alexander was talking about we need to foster more democratic societies. I think we also need to foster more inclusive societies and I think the work we do is actually contributing, I hope, to that. So we make sure that in every team there is diversity of backgrounds, inclusive tech refers to mobile first because I think 90% of our users come in via mobile. Download speeds are fast. Inclusive content is also bridging all kinds of democracies and ideologies, so make sure that content is being balanced to not end up in an echo chamber, but that's always a challenge. These are some examples of media formats that we use. The first on top left is a blog, so we work with highly regarded blogging platforms in Mali, Buruni and DRC, so they use persuasive storytelling. The bloggers use a lot of also personal stories to, for example, talk about sensitive issues in here. I think this one is about genital cutting. The blog is called Letter to My Lost Cliters, I think. Yeah, that's the one on the right. Yeah, exactly. So that one is an example of an infographics done in Yemen, so where the community was asked to give their opinion on cheating. So these are information is collected via surveys and then fed back into the community, into using all kinds of infographics and forms to visualize that data. On the left is Amani, our first female vlogger in Libya, and she started talking a lot about issues women face in Libya, for example, on the workspace. This creates huge kind of debates, also sometimes very violent debates, but those are moderated and we see that the tone of voice is changing over time, so that is, I think, great progress. And then on the left, right, it's an example, it's a still of a motion design video made by our team in Buruni, and it's actually, it's an explanation of a prejudice or stereotype around people from Mwado, which is in the center of Buruni. I think it's quite sensitive, they are called to be greedy, and the video is made together with a historical institute in Buruni, where they go back to in history where this actually came from, so also trying to bust that stereotype. Yeah, moderation means different things to different people. For us, it's actually used to increase the debate, to deepen the debate also and to increase also participation of certain groups that are underrepresented. And this is a nice way where you can see that the average Facebook engagement of women is actually around 10% while our platform is much higher, so I think that's a nice example. Okay, let's run into the impact results. So what did we do? We did a survey in five countries. In total, almost 1,000 answers were analyzed, and the question that we asked was an open question, so can you give an example of a way in which your perspective or your opinion has changed through reading or engaging with the platform? The methodology was, yeah, we are all practitioners, so I did this together with my colleague from Yemen, who I would love to have on the screen, but unfortunately, that was not possible, because I wanted to Skype him in. Colleague from China and from Egypt and from the Netherlands. So we did qualitative coding. We made sure there was a second coder for each project, and we found a core group of similar kind of types of change, and the types of change we found were actually much broader, I think, than we expected, and there are specifics for each country reflecting their context. So we found that 84% actually reported a change, and the biggest change they reported had to do with more openness to different viewpoints. There's also included answers that we're talking about, that they're now less prejudiced, that they now more aware of discrimination and how to act against that. Maybe that has to do also because some of the platforms have quite a lot of content about a marginalised group, for example, in Yemen, about a more machine, in Libya, the Amazigh, in the RC and Burundi, there's a lot of content about albinos. Knowledge turned out to be a big change. Some people reporting, we now have more access to either more trusted information or different information, or pluralistic information. Sorry, and then have increased knowledge around these kind of topics like political situation, minorities or gender issues. A smaller group reported behaviour change, they were talking about they now are better able to express themselves or more open to express themselves, or skills having more improved communication or analysis, critical thinking skills. So a little bit into the different types. So the first one, access to pluralistic information, I think that's where everything kind of starts, by having access to information, you can increase your awareness, you can change your attitudes, and these are some of the answers that we got from the different countries. Then, more awareness, so this is also reflecting to the, I think the right image actually doesn't reflect the answer, but on the left you can see the Mwado video again, so the people talking about how they now understand, for example, certain cultural issues better. Attitude change, these are examples that we got from, yeah, for example, the person in Buhuni is talking about how he or she is now realises the importance of listening, and that also, I think that was also coded as a better communication skill. More open to express oneself, so this is not only actually being more interested in, to share opinions or feeling more free, but there was also people saying actually that they now feel a need, a bigger need to express themselves where before they might have not spoken up, they now see they should and they will do. Then, some specifics, so Buhuni had a few categories like improved communication skills, so people saying that I used to be, I used not to be able to have a conversation with someone that I disagree with, and now when I see the bloggers, how they kind of speak, and also the moderators, actually platforms are heavily moderated, how discussions are kept in a constructive way, this actually also reflects in their answers. And another, I think, nice one is a more positive views on Buhuni and people saying that, yeah, for example, talking about entrepreneurship, creativity, so there's a lot of content on people like Diffin on the picture, who is a portrait artist, so very, very inspiring and aspirational content. DRC, yeah, this is an example of improved analytical skills, so people saying that they're now, but are able to address situation from different angles, and that before drawing conclusions. Also, people mentioned that they now check several sources or that they don't trust the first thing that they read, and they go look for more information on a certain issue. And in Yemen, yeah, like I mentioned, there was this quite a lot of attention that the team there paid to more machine and also changing stereotypes around that group, and that is also reflected back in the answers. And then China is a bit special, China has a different kind of target group, the people there that we have on our platform are female migrant workers, and the content focuses on gender-based violence and also worker rights. And there we see also more, I think, more nuanced kind of answers, people talking about being more aware of issues, but also a smaller group saying that they now understand that they were not the victim of, it was not their fault, being a victim of gender-based violence, which every time I read it, it gives me goosebumps. And another quote from that sample was actually an example where someone realized that domestic violence is not normal. So it's actually a restrictive norm that actually is being, people are being aware that this is not normal. So it's kind of a first step to change your attitude around that and to maybe act differently. And I'm not sure if I can go back. Yeah, so here in the China sample, we also saw people talking about they're now would act against violence, or they would stand up for others who actually are a victim of violence, or they would say no to harassment. So it's kind of showing a kind of behavioral shift. But yeah, I don't know. I didn't kind of, I didn't want to conclude this. So this is my final slide. I just wanted to ask you, but we don't have time anymore for the question. But maybe we can talk about it afterwards. So how to give meaning to this. So because, and there's also some questions that I had for you actually prepared, but we don't have time also for that. But maybe later on with the drink. So I would love to hear more about which experience you have tracking offline impact of online interventions. And what other quantitative methods you see to measure to demonstrate changes in attitude and behavior, given that our user base constantly changes. So people aren't around for quite for a long time, maybe a few months, maybe a few times only. So it's difficult to do a baseline and line kind of measurements. Yeah, that's it. Thank you. Thank you. First, it sounds like you found the secret source to engaging young people. And I'm sure there are people in my society who would like to hear more about that. Next, I'd like to welcome to stage Luke Jordan. It's the that session where there's the coffee only just arrived. And it's just it's like an hour and a half after lunch and everybody's falling asleep, including me. So I'm going to do this walking around thing with the mic. Because if I'm at the lectern, I don't trust myself to stay awake. Presentation. Okay. So grassroots. And my apologies for people who are in the session this morning. Some of this will be repetitive. But as a brief sort of summary, grassroots is when organization in South Africa, we deploy mobile first tools that enable community organizing via pretty much any kind of phone from feature phones to low spec smartphones with limited data, through to more complex devices when people have them. We were launched in late 2015, early 2016. And we've now reached over 350,000 people with over 30,000 activities called through it. And that rate continuing to grow at about five to 10% a month, almost entirely through organic word of mouth. On the surface, and this will be the main subject of the talk, kind of on the surface, our metrics look very strong. So we did surveys of users. When you ask when we ask users, can you talk about changes that you've seen in your community? We have two thirds of users saying they've seen a significant change in quality of life. When we look at things like viral metrics, how likely are people to recommend the tools to other people, our scores on that are near some of the best in the industry. And we have stories of first level impact. The challenge comes in. And I noticed that there's going to be an aspect problem. So half the tech some text won't be visible. But we've in the last six months, as we started to dig quite deeply into what's actually happening in a way through really quite forensic qualitative research. We've actually found that as good as some of the surface metrics are at a deeper level, some of the impact is not really flowing through. And what that means is that our theory of change, which I'll talk about is quite probably broken. And that leaves us with a question, which is, okay, so what do we do now? When we've built on a whole bunch of assumptions, which turn out to be wrong, and it's not clear where we might go next. So here's the context and I'll go through the first slide a bit quickly and then dwell a bit more about where things are failing. So the problems that we're trying to solve are first, it's very costly and difficult for the poor to organize themselves. And one example of that is in South Africa, if you wanted to hold a community meeting, you had to hire somebody to drive around in the back of a truck with a megaphone telling everybody when the meeting would happen. And of course, most people wouldn't hear that. And a lot of people would would not turn up and it's expensive and time consuming. And those can be multiplied. And the second thing is that it's too hard for local leaders to access information that they can act on and also to connect to each other in a meaningful way to start aggregating and trying to create systemic change. So why do we resolve them? Sorry, the text is all messed up. But anyway, first, South Africa does have the full panoply of institutions that democratic theory says, once activated by engaged citizens should lead to responsiveness and accountability. I mean, we have the entire works from formal elections through to independent press, independent judiciary, we have performance monitoring and bureaucracy, we have legislation for participatory budgeting, we have legislation for participatory performance targeting. Every single thing that according to formal theory should lead to accountability. If people activate and use it is present. Second, South Africa has a long tradition of collective action and the struggle against apartheid. And today has extremely high levels of dissatisfaction and anger, resulting in some of the highest levels of per capita protest, although often disaggregated and small scale protests in the world. So we had, and this is what led into sort of there, we had these two things saying, well, on the one hand, we have all of these formal institutions, on the other hand, we have all of this grounds up anger, we can channel one into the other through collective action, this should lead to significant change. So in order to pursue that, what we did was we created simple tools for mobile phones. We designed very closely with our users. So we are half fieldwork, half software development. We built a set of tools that allow people to do things like recruit a whole lot of people, call a meeting, ask for volunteers, call a vote, issue a press alert, issue a safety alert, etc. And to do these things, using very simple menu based flows on any kind of phone. And we did that through spending months at a time understanding a problem before we touch a line of code, and by having a field team that fed back very directly into our development team and vice versa. So what that led to was that after two years, we had a platform that without pretty much any marketing budget or much marketing at all, was able to achieve quite rapid increases in engagement and quite sustained increases in engagement. So somebody this morning talked about sort of initial bumps of use and then fall off. We've seen a kind of continually climbing curve. About 80% of growth has been driven by word of mouth. And people would report things like twofold increases in meeting attendance and frequency in communities using our tools. And we'd see over 2000 activities a month flowing through that's meetings, votes, etc. across our user base. As an example of stories. This is Abathlali Basim Jondolo, which means the people who live in Shacks. One of the most vibrant and I think widely known social movements in South Africa. They grew from 14,000 to just over 50,000 members over a period of three years. They use grassroots for quite a bit of that organizing activity. As I mentioned later, I would have mentioned in Q&A they give us more credit than I think we take for what they've done. But they use us in ways that help them organize at that scale and grow on that basis. They are currently involved in they are being targeted by oppressive violence and various other forms of repression. But they are a strong and growing movement. Overall, that curve is actually already outdated. But sort of activities being engagements and activity has been growing really rapidly. And we've now started to be used in large scale campaigns. So part of one of the set tools in the suite is the ability to do change.org or move on type petitions through very basic phones and connect that to new methods of awareness raising in order to trigger large numbers of involved engagement and then convert that into things like WhatsApp voice notes and other media that can then be brought into policy spaces. So all of that activity has kind of been undergoing and happening. And actually, just last week, various things happened that released result in another 30, 40,000 people coming on. So that's all happening. But then dealt. So and this is where I'll spend a bit more of the time. In the last six months, what we started to do because we started off as right now, it's about three and a half, three, three and a half years ago. And throughout the initial couple of years, we kept saying it's too early to look for deep impact. It's too early. It's too early. We're working on changes in collective action, which are going to result in changes in governance, which are going to result in changes in outcomes. That's a very long time horizon kind of activity. That measurement is quite difficult. So it's too early. It's too early. It's too early. And in six months ago, we started saying it's not too early anymore. We need to go and start looking at what's actually happening. And we had the survey result, but we said for various reasons, I guess it's just congenital self delta pessimism. We, we just decided, well, we're not entirely sure if we trust us. So what we did was we went through and we conducted interviews with just over a dozen of the communities that have been working with us. And when I say forensic interviews, I mean, three, four hour long interviews, really trying to understand in great detail what exactly is happening, what are the dynamics of organizing. And in all of them, what we found was a similar cycle. So what we found is that there would be an initial phase of collective action increasing and picking up. There would be some initial response from the state. The response would stop. And then for various other internal reasons, the collective action would fall apart. And this is a cycle that existed before we were there, of course. But our, our tools and what we were doing was not breaking that cycle, it was accelerating it. So it might be increasing the amplitude and sort of decreasing the frequency, but it was not actually breaking the cycle. Which means that an increasing density of organizing is not leading to effectiveness, which is the link that we have presumed. And of course, just to be clear on realism, you know, we didn't assume like, oh, we'll build this USSD meeting call and everybody will sing kumbaya and the government will start responding to everybody. It was we always realistic that this was a case of shifting at the margin. And that maybe we'd see sort of where we had where there had been kind of nothing had been feeding through into response. Maybe we get 10% maybe we get to 20% or 30%. But unfortunately, what we're seeing is actually somewhat more serious than that. And why we're seeing it as we've dug down is and this is still a process that's ongoing. But we've seen three sort of interlocking causes for why, despite massively reduced transaction costs for collective action, and a clear motivation to do it. Still, there's not this feed through into impact. And the three unemployment institutions and ideas. And there's quite a lot in here, but to kind of talk through each of them. The first one is South Africa has 30% unemployment. And roughly 40% of the population is dependent on income grants. So as an aside, anybody who thinks an automated future with basic income grants looks good, please come and look at South Africa. Because we are that future to significant extent. And what happens in those conditions of unemployment is any additional income opportunity is extremely valuable. And being a community leader creates those sort of opportunities. And it can do some ways that are not directly corrupt. They're just in the nature of operating in that sort of environment and having that kind of precarious, highly resource constrained life. So when and this particularly happens when external NGOs or government departments come in and run projects in these kind of communities, it's well known that when that happens, whoever can plausibly say I am a community leader, is going to have access to certain kinds of income generation. Second is when you have that level of unemployment, you have long term planning and thinking becomes very difficult to entrench. It's not difficult to do, but it's difficult to make a habit. And it's also there are few productive channels for energy because people generally have very few productive activities. And so that energy often turns inward into suspicion and dispute. And then that's exacerbated by the fact that any source of income, because no source of sources of income are formal, any source of income as it can be a potential cause of suspicion. Institutions. So South Africa, the institutions of local government, particularly despite all of the laws, it's almost a case study of how legislation can be entirely meaningless, or close to the structure of how political powers distributed and allocated through constitutional provisions around institutions around control of powers around electoral cycles, means that almost universally most of the provisions are ignored. And unresponsiveness is basically structural. Where that particularly means in practice is that there's an occasional response to a first message and none thereafter because all of the incentives that any local bureaucrats or politician faces are all inward facing. As one of our community leaders said it, they look at your letter and say, Oh, you have money to buy ink. That's nice. I have money for a dustbin. The final point is that within ideas within these community structures, there are not that many new narratives or ideas of what it means to actually be on these communities and committees. So anybody familiar with Marshall Ganz's work about public narratives and stories of self and us and now, there's very few of those kinds of stories besides being on the committee, which is a very shallow base for really taking forward deep collective action of the kind required to create change. What ideas and narratives they are come from, frankly, very archaic Marxist academics who still live in the South African kind of university ecosystem and run sort of workshops. So you have a lot of community leaders who can't really talk about current challenges or sustaining that, but can talk very eloquently about sort of means of production and class struggle. And then finally, officials and counselors often have a lot of discursive tools to render things illegitimate. Often this is taken from technocratic discourse around things like data and so on. So it's very much like the only there are whole kind of projects that basically exist to try and prove to officials that an informal settlement that has no toilets would like toilets. And there's a whole kind of thing of sort of people say, officials saying well, your data is not representative. How do we know that actually this community that has no toilets wants toilets? So where does that leave us in terms of our road out? We're currently not sure. We know that in terms of unemployment, some micro practices, details of how the committees are structured, how meetings run, how committees are elected, can actually contain and channel energy and enable people to overcome these problems. But so far, the evidence we have is a bit anecdotal and possibly only in some context, but we're investigating it forward. Institutions, we are running a campaign at the moment to enable recall provisions for local councillors to at least start to break some of those incentives. And then after that one move on to something like citizen juries or sortition or other more radical changes that would actually require pretty deep restructuring of our constitution, which is both a perilous exercise and something where we're not sure if any social force we can see has the energy to make that possible, but we think it's almost certainly necessary. And then under ideas, there are forms of sort of what we call pedagogy, deep training, idea development that might be possible. Again, there are sort of activist schools that we don't think do very much, but there might be new ways of say developing things based on WhatsApp that are based on group discussion, media distribution that might have a shot at that. So overall, what kind of coming out to say we probably need quite deep reimagining in practice of what the constitution of a democracy looks like in South Africa and a high risk large scale attempt to conduct new forms of pedagogy. I will say that some of our funders have been amazing in their ability to withstand us going through this level of self-rigger, not all. So there's some question around both that and kind of wider patterns of discussion of having the tolerance to go through these sort of exercise and try things that are very difficult and different and new. So conclusion, and that is, you can't read that text, but anyway. So I mean, there's a kind of, there's a side of the story which says rapid sustains and your viral engagement is possible if technology is built well and large numbers of people may consider that technology useful, but even uptake and sort of people self-reporting usefulness may not be immune to rigorous self-query even with a theory of change that's articulated on some foundational ideas of democratic accountability. Some responses to this might be a narrowing of ambition, kind of find something smaller that you can measure and report on and so on. We don't say that not always, that in general that's a problem, but it's just congenitality, I think, temperamentally, not something that we find as a possible or viable or attractive path. But in something that's much more ambitious and wide-aranging, there is some question of kind of what is the role of civic tech, if one is considering the movements to restructure what democratic constitutions look like in South Africa or elsewhere. Is that a civic tech project or are we jamming tech into something that's actually fundamentally very political? So we don't know and I guess I'm just going to conclude by saying, and this was a phrase somebody gave me, a mentor gave me recently, there's no real conclusion here of the form that you see a bit often these days, which is kind of inequality is soaring, the planet is crumbling, the political class is decayed, democracy is under tech. So hey, let's pass an antitrust law. The roads out are pretty narrow and difficult and unclear, but that also might just be because we haven't thought it all through. So I guess we don't have Q&A, but during drinks and coffee and everything afterwards, if anybody has any ideas for what can lead us out of this path and also the kind of experiments that we're starting to run, any advice for those, we'd be more than welcome to hear them. Thank you. Thank you, Luke. And finally, Natalia Dom Maggala. Massacring names all over the places this week. Hello everyone, thank you very much for staying with us for this session. I just want to start with a short disclaimer. This is going to be slightly different from the two previous sessions that you've just seen, because I don't actually work for any of the organizations I'm talking about. So I entered this space as an impartial researcher and I will share my findings related to the empowerment of women through open data and civic tech ICT skills programs. So first, let's start with the obvious. It is common knowledge that empowering women, sorry, creates wider socioeconomic benefits for the countries. And technology is often seen as a tool to support this empowerment. However, globally, women have much less access and control over ICTs. And this leads to them being economically marginalized. And if this digital divide is not addressed, the problem will unfortunately get even worse. Now, digital skills and technology do have a potential to empower women and create high-skilled human capital. The question is, to what extent and how do we do it? So the projects I looked at are based in Kosovo. It's Girls Code in Kosovo and Open Data Kosovo. Girls Code in Kosovo started as a grassroots initiative because the founder, a woman working in the ICT sector in Kosovo, Blair Tatzi, saw that there is a big gap between the number of women who attend ICT studies and the number of women who actually work in the industry. So she was curious why is that and whether she as a woman working in the industry can do anything about it. So initially, it started as a very informal space for women to meet over coffee and just discuss their challenges. And for the past two, two and a half years, they have actually been running ICT skills programs with the use of Open Data. And some of the programs they have conducted are, for example, tech for policy, which takes groups of women developers, teaches them ICT skills, and then purse them up with their local municipalities to design digital solutions that are needed in their cities. Another project is code for Mitrovica, which brings young women from Serbian and Albanian sides of the divided city. And using ICT skills makes them design solutions, digital solutions for their city together. So my overall question was, again, to what extent do such grassroots ICT skills programs actually empower women in Kosovo? Now, empowerment is a little bit of a buzzword, isn't it? And I was looking at it from a very academic perspective. And some of the definitions I found were, OK, like this one, but not necessarily very practical if you want to look at it from a researcher's perspective. So having established that empowerment is, first, a very fuzzy concept, second, a multi dimensional process, very relational, very context dependent. So what empowers one person doesn't necessarily empower other people. And it constantly changes, constantly evolves. So what I did was merging of two frameworks that I found in the academic literature, tracking the types of empowerment and the process. And in my actual research paper, I looked at all four of them. Here I will only focus on the political slash civic empowerment, which basically in the literature, the knowledge that I found is that ICT trainings can provide information and bring women together, thus supporting their civic empowerment. However, I know for us being here at the civic technology conference, it's pretty obvious that there is a very explicit link between civic and technology. However, that's not usually the case. So one of the challenges is how do we actually engage the ICT, IT communities into civic and political engagement. And again, many tech-enabled communities have been criticized in the academic literature as disconnected from governmental practices. So you might ask why focus on Kosovo out of all places, right? Well, the answer is that Kosovo is a very unique case. It's the youngest country in Europe, both demographically and historically. It's very poor. It's currently the first country in Europe. And then it has many problems. So it's a very young democracy, still relatively politically unstable. It has a big problem with youth and female unemployment. And the challenge in Kosovo is how do we actually use the workforce and the skills that are present, considering that there aren't really natural resources. And people from Kosovo have trouble going abroad. It's very hard for them to get visas to go anywhere. So basically, the country is very problematic. That's why I think it is a very good example, because here, ICT skills, civic tech and open data can really change things probably more than in other contexts. So methodology I used was qualitative because, as I've already said, empowerment is quite a difficult concept to measure, particularly in a quantitative way. So I interviewed people from girls coding Kosovo and open data Kosovo. And what I found, which is I will come back to this later, but basically, five of the organizers who currently work for either of the organizations used to be participants, which I think is quite interesting as well this small scale retention. So what I found is that unfortunately, disempowerment of women in Kosovo is really prevalent. So here you have a few key points. So traditional divisions of gender roles and cultural constraints are unfortunately the biggest challenge in Kosovo still. Women are actively discouraged from working at all in the most extreme cases and particularly from working in the ICT sector. So more than half of women I spoke to were explicitly told that ICT is not for them because there are women. Once they get into the industry, they often see their skills being undermined. For example, I spoke to a very young 23-year- old woman who developed her own app. And she was presenting her work at a conference with her co-founder also a very young female programmer. And they were asked by a group of men. So where are the guys who developed this product? And when they said, well, it was us, we made this, we are programmers, then the men were really skeptical and they kept on asking them really specific questions that only someone who actually developed the app will know. So this is very direct discrimination that they have to face. There is also subtle discrimination. So once they do get into the industry, they often get allocated administrative roles or softer, soft skills, more artistic, let's say, roles. For example, a few participants, my research participants complained that it was just assumed that they would deal with a design and front-end development rather than back-end development that they actually wanted to do just because they're female. And as a woman, you're supposed to have a preference for aesthetics and design, apparently. So these are the three key problems. Another thing is that in Kosovo, women have a relative lack of exposure to the ICT in contrast to men. So they're not socialized because of those cultural constraints. They're not socialized to engage with ICT from a very young age the way that men are. Now, a very briefly technological empowerment that I found in my study is basically the fact that women gain skills that are very, very desired in the labor market and are really hard to get because as my participants complained, the quality of higher education in Kosovo is not great. So courses like the courses provided by Girls Code in Kosovo actually bridge this skills gap between university and industry. And again, they employ some of their participants, which creates a small scale labor market empowerment. They also match some of the participants with available jobs in the industry. Courses are free, which I believe is really important, especially considering that most of the women are young and are still trying to launch their careers so wouldn't necessarily have the funds to upscale. And the participants become fluent in OpenData, which is something that Girls Code in Kosovo and OpenData Kosovo use a lot. And in terms of social and psychological empowerment, just really quickly, the fact that there is this tight community of women really helps counterbalance the unfavorable social norms. And they gain motivation, confidence. Blerta, who's the head of both organizations, she is a real role model to the younger women, which I think, again, shows them that one day they can be in her space. And some of the women I spoke to were telling me that getting enrolled on those courses was actually a real life changing experience, because having graduated with an ICT degree, they couldn't find any jobs because they didn't have relevant skills. And they were losing hope, and they can't leave their country because the visa situation is tough. So actually having an opportunity to gain these kind of skills for free in a very welcoming space is something that did improve their mental well-being and quality of life overall. And finally, civic and political empowerment. So for data-driven projects in the civic tech space, OpenData Kosovo and Girls Coding Kosovo create a platform for young women to engage in their municipalities in an active way. So they become the producers of technology rather than being the consumers of technology. And some of the women I spoke to coming from the ICT background said that they never really had an interest in their local affairs. And through getting to use OpenData, through getting more engaged with this transparency agenda, they actually became more involved in what's going on in their municipalities. So for example, one of them said that before she didn't even know that her municipality has a website and that there is anything she can do for this website. But after the programs she actually went on the website a few times and because she thought it was quite badly designed, she offered that she can just redo it for free just because she thinks it should be more accessible to further engage this kind of participation. So what I discovered in the case of Kosovo was that OpenData became quite an important link between the ICT communities and the civic communities because it gives people from the tech industry a reason to actually care about what's going on in the civic space. So there we go. Just bringing women together is basically insufficient in some cases. And here, the fact this active role of women, the fact that they actually got to meet their municipality officials, that they sat down and discussed together what is needed in their cities and what kind of digital solutions can be made. This was really helpful in terms of showing them that they can play an active role in this space and that they can have a more important role in the industry and in the local civic community. Again, although they are doing a great work as an independent researcher, I also felt that this is my duty to assess some limitations of those programs. And really quickly, the empowerment happens only for a certain group. So the programs are mainly for women who do have some sort of ICT skills background. It's not exclusively for women with this kind of background, but that's just the way it usually is. And the courses are advertised on social media, which also means that it attracts women who are already empowered enough to actually have access to social media. So again, there is a problem with targeting ethnic minorities here. So empowerment does happen, but not for all. Now, programs are complementary to existing skills. It's not required to have the ICT skills background, but again, they're usually quite advanced the issues that are being taught. And the employment assistance they provide is quite limited because there isn't really a scheme or an internship program associated with girls coding Kosovo and open data Kosovo. It's basically if they hear of some jobs that are relevant, then they help and match women with those jobs, but it's not as big a part of a program that it could be. But again, perhaps that's something they're looking to develop in the future. And in terms of open data, which is quite a big tool to stimulate this interest and civic engagement, in terms of the general public, the lack of it and the lack of the knowledge of open data is really is really poor and the awareness of it is quite low in Kosovo overall. And again, as much as women become more interested in and empowered within the civic sphere. So for example, fixing what's wrong in our city that doesn't necessarily transfer into their higher political engagement, which I think in countries that are like Kosovo young and relatively unstable democracies would be quite good if there was a way to link that to higher political engagements. And finally, this is a rather complicated diagram or two diagrams that I made in order to map what they are doing and what kind of benefits and types of empowerment it creates. But in a nutshell, what happened in Kosovo is combining technology, people and open data for a broader social good. So the main problems that they had prior to starting the programs was the lack of digital infrastructure in governments, particularly local governments, having a large group of women without the appropriate skills and perhaps with relatively low interest in civic matters. So what they did was to train this community of women to actually be able to build the digital infrastructure that the cities need and get more involved in the civic and political matters. And there are a few overall policy implications that I think could be used in implementing similar programs. So first, open data and ICT skills and training should be deployed as complementary tools for upskilling their workforce. So if there is a really good civic tech ecosystem, then open data is not necessarily as important. But if there is a gap between technology community and the civic political community, open data can be a very good and effective bridge to actually connect it to. And in the case of Kosovo, the ICT skills, trainings provided a very neutral space for people from two communities that historically don't necessarily get on. So the program Code for Mitrovica brought women from Serbian and Albanian parts of the city together in one room, working in groups to think of what are the solutions they could develop together to help their city. So again, this is something that could be used particularly in the case of younger generations in some spaces with community tensions. But again, it's no silver bullet, so don't take it as gospel. And in order for this kind of programs to benefit long-term, to produce long-term benefits for women, unfortunately, what we have to do is to overcome the cultural constraints and the subtle discrimination and the direct discrimination and unfortunately, we can organize as many upskilling initiatives as possible, but without a broader and long-term culture change, I don't know to what extent it will actually empower women. Thank you very much.