 My talk will be about map swipe and I always like to ask that question first Who of you has already heard about map swipe? That's great. And who of you already contributed? Great even better. Okay, so hopefully the rest Who did not already? Contribute just come out afterwards. We have a short demo session as well, and they can also get the app All right, so but what I want to present today is like insights into the app and also into the data quality that's coming out of the app because Yeah, that's really interesting how useful it is and so just for those of you who haven't seen it so maps wipe is a mobile application and it's designed to Classify satellite imagery and the task that we've been working on most is finding buildings on imagery like this So and every time you spot a building you just tap on your screen and indicate Yes, there's something so it turns green if you tap twice you can indicate. Oh, maybe I'm not sure Maybe this is a building, but you can also say I know he is no imagery at all or their clouds by tapping three times So this is the app the app was developed by MSF within the missing maps project and we are going to hear about it more in the next presentation so Yeah, maybe just To clarify two things when I talk about tasks I always mean these tiny little squares and then I will also talk about groups and groups is what you will work on It's like a session of tasks so you start mapping these six then you swipe to the left and six new tasks will pop up and in the end you will somehow reach 100% and upload everything and These tasks put together as one group. So like just to make this clear because the next slide I will refer to these words again and again All right. So why are we doing this? So we had the problem of how we would like To find all these buildings and huge areas, but most of the times they're also sparsely populated. So it's like really Scanning through areas where there are no buildings and to do this faster and faster. We tried out the map swipe approach so we have a huge area of interest Generate these tiny little tasks and then have these crowd sourcing approach and every task will be evaluated by many people So at the moment we have at least three different people looking at everything and saying oh, yeah, there's house or not And in the end we can aggregate everything and derive a settlement layer for example So then we have like the aggregated information like just one layer easy to use and showing us where people live So that's the goal and that's the workflow here But now the question is how good is it? and we did a case study and So we chose two projects from map swipe one in Madagascar the other one was in South Sudan and This also just to give you a brief overview. How is these projects look like? So they cover quite huge area. So that's more than 6,000 square kilometers and Resulting in 1,400 of these groups and that's in the end about 1 million Contributions by the volunteers and for this projects about 850 people have done. So that's quite a huge amount of people and We chose another project from South Sudan. It somehow has a similar size and Also similar number of user contributed and what we entered in is now How good is this data? So because as you may know People are not experts in looking at satellite imagery. They don't have a GIS degree or geographers They're just people like you and me and that's also good, but that's why That's why we want to know how good is this All right, so know a bit more technically Mmm, we looked at how people agree on On their dancer. So that's what's the inter-rater reliability is about So we have this group and we look for okay How is the inter-rater reliability and what we can see is that all right there for many groups It's quite high. So like More than 0.5 is a value where we can say that's a good agreement But also like that's somehow present for those projects like if we have 1.0 agreement of one means total agreement everyone said for each task the same thing So that's really really good, but also rarely occurs, of course And but still we have some of the groups where we have really low agreement and we wanted to understand Why is that? So and even for the South Sudan project. This is even more present. So In ideal case, we would like to like yeah cut down this portion here and edit somewhere here All right, so Let's see at two maps. So that's maybe the first time You when you contribute you can also see like how the data looks like really like not only these tiny squares but everything put together so Green means good agreement So the areas where it's green we are really happy about so people saying the same and This is the indicator of good quality as well. So we analyze that before it's like when people agree. We can almost Be to 100% sure that they found the correct answer that there's a house or not if they don't agree That's also for us difficult to handle. We don't know what to do so But let's have a look at these pattern and maybe the first thing that we can see is that They're not random So they somehow tend to cluster. So that's what also these more and I Index is saying that it's not a random distribution of disagreement, but it's clustered. So there are specific cases where people disagree and So for example, we have these now in our south bands, but we have also something maybe here from east to west and Now we are asking ourselves or what are the reasons for this and we did more Like to get out the proportions of disagreement. So for Madagascar, we had a really the most Disagreement cases where between no and bad so some people said no, there's no house The other said that's bad imagery and this was the most of the times caused by clouds better We are present in the image imagery and people didn't know what they should do now Is this like is there no house or is this like bad imagery? and so this already gives us a good understanding of what we can do better in the future and Maybe to address this then we could also get a higher agreement and better quality Mmm The same like it's a bit different for the South Sudan project here We have a really high disagreeing between yes and no so this may be cause because the Houses that are present in symmetry. They are maybe really difficult to spot. They are maybe wound They have a similar color than the ground and that's why there are some people who are able to spot it and others They aren't and what we would like to do to improve this is maybe to give better guidance Provide better tutorials so that it's easier to classify these kind of satellite imagery So this is what we learned from the South Sudan task mmm and What we then also see we have now the these different cases mapped again here and What we see is that also these pattern are associated with different kinds of this agreement So for example these north to south bands. They're Almost every time caused by missing satellite imagery So if we spot something like this, we know what went wrong and that's good because in the next step We can address it directly. So it's not not not a miracle for us anymore Also these East West things that they are Mainly caused by individual users who unfortunately did it wrong and most of the time that happens like just after beginning your first map swipe tasks and Soon they get better, but still it's present in the in the data and we need to get rid of that part All right, so next steps So we have I presented the analysis for two projects at the moment. They are more than 50 So 55 projects are already completed. I think three or four projects are active So we would like to do the analysis for all projects because they may differ between these projects At the moment, we only look at agreement among volunteers we would like to add further parameters to have estimations on the quality and That's also Something I would like to do like update the Tasking and map swipe dynamically. So when we notice that there's a low Agreement, let's do it again and again at more people have more people looking at it because that may increase the quality But if we already have a high agreement, then we can just finish it and say, okay, that's cool Yeah, and finally Something that's of course many of you may be also heard about is that there are many good automated approaches to detect objects and images and We would like to try to combine data from upside with these approaches to then improve the quality again Because that's what we want. We want to be really sure that our settlement layer is correct and that we don't miss information So thank you for your attention Have a look at the website where you can find all these indicators that I presented you and Yeah, thanks So, thank you to Benjamin. We have a 15 minute session at the end rule We'll have all the presenters on the stage to ask your questions But if there are any short questions of clarity that we'd like to ask now, then please raise your hand And so you have one on the left here on my left Hello, my name is Courtney OCA MSF could you maybe briefly Explain something about the real time of these type of data considering that MSF is also using in areas Where there's a lot of movement of populations so so I think by real time you mean how How reliable is the information on the settlement and that's the most critical issues like the Temporal Resolution of the satellite imagery we are using so if we are using satellite imagery from 2010 our data is as good as it was in 2010 so this problem is mainly do what kind of imagery can we use if you can use really recent imagery We can really do it good, but if we have to rely on old imagery that maybe adds further Limitations I will have a question. Yeah, I do and I had a question on the way the tiles are cut Do all the people look at the the same tiles or are they spread differently over the map? No all these small tiles there's like a standard tiling system For example, if you open street use open street map or Google Maps or when you load a page They will also pop up these tiles. So that's the standard grid. So there's a grid. They will not overlap So it's really unique One further question. Could you state your name and institution, please? Clare Mills OCP, and I don't know anything about this So I've been back two weeks in MSF, so excuse my ignorance, but why would we use humans with all their known variability and errors over Something that surely could be further computerized. I mean couldn't we get Some way of looking at this in a more smart electronic way Mmm, so the interesting thing about these automated approaches is that They will work for really good settings. So if we have a good imagery if we have a clearly Identifiable house with like rectangular shape and the red roof for example, that's easy to detect But in most of the areas where we are doing maps right even for us as humans It's really difficult to say is this a house or is this a tree and then it gets worse using automated approaches So I think at the moment we cannot Solve it automatically to a degree. We would like to do so reaching 80 percent automatically is maybe feasible But we still missing 20 percent And that's not feasible for our approach then One more quick question Hello Clara MSF, how is it it is to get access to the satellites imagery? so Basically, it's I would say easy like it's the same approach. We also use an open street map So Bing is providing imagery for this kind of task at the moment So it's like just adding it's like just a link actually just yeah, not more than that