 Five to ten minutes Question answer discussion if you want and I would invite you all to actually do that make use of it for some it's the usual Geospatial topics of course but others are new to it and might have other questions It's nine o'clock the stage is yours. Thanks. Thank you came on early in this morning And the first presentation today is about how we can use them deep learning to improve data quality from a data set and here we took OpenStreetMap as an example So the goal of the presentation is to show all we can detect inconsistency between two data sets and between these two data sets we can Highlight where there is some some issue that you might have to check To to verify if everything is online. Yes or no, all we do that with deep learning Ecosystem first is to to to bring imagery and the related levels And to train a neural networks with a loss function Till we get a train model able to use the very similar kind of imagery to predict an answer and as soon as we get a train model and the ability to to product Imagery we are able then to compare between The prediction and the alternate data set in our example. It will be here OpenStreetMap So that's the main ID to train a neural network to predict and to compare that the point What we use to to do so we took RoboSat that pink with Semantic segmentation ecosystem able to deal especially with just a cell imagery and to perform some Main feature as a data set quality analysis chain detection and feature extraction So that will be the software some the tools are able to to help us to do that What the spirit about that is to bring the latest? Semantic cementation a set of art available in in academic field, but to translate it in a neutral or robust code and to allow by a modular design to Extend and to to write only support you could want to to To make it your owner And the other point will be the ability to be integrated well integrated in either OpenStreetMap or mapbox ecosystem and to be a special compliant so what Cool, how could it help us to to make it work first is to It's to help the data progression part. So as since we have several tools able to To save us time on the data preparation. It will be really helpful the first is the ability to use either raster or web services To create imagery the other is the ability to use them jorizen or OpenStreetMap Files to create the labels and I think we we got the boss imagery and label We are able to create our own training data set and so to play with so the first Interesting part is to help us to to create easily Training data set the other Interesting part is the ability to reuse already train model who came from from image net There is a lot of model already able to To classify To classify a feature from any an image so this is The internal representation of the of the model With the first part with the encoder Who took an image encode it and class every pixel? Among the the output feature we want and the other part is the decoder and the point here is to reuse Encoder pre-train model to help us to be very accurate on the kind of classification we we want and in this really an example We reuse ResNet50 as an encoder the third The third part is the ability to To perform accurate Training from the co-poll so here the image and the label are the input co-poll and If we use a really classic Lost function we get this kind of output. It's a bit noisy but if we Use another kind of loss function as this this to one one far more on a Design to to take in in charge as a semantic of Of the feature we can As a direct output or got something really more accurate and without all the noise That's a classical loss function could bring us and so the point here is to to have something sharp enough to Avoid the need of a post-process on the on the output and as you can bet What we use here in Robocat Pink is a Lovac semantic list The other point and I think we get the ecosystem able to to deal with with data and to train it in in order to get a train model is what kind of data could we use to Train them to train the model The fact is we don't have at this moment really Reliable open data set for Geospatial data because all the label we can we can grab are some some parts of Creepy or not that accurate and one point is to say okay, how could we bootstrap? a good open data set The point here is to say okay, we can reuse open data to do that as since we are able to filter them So, oh, how could we do that? By reusing imagery from an open data in this example I took an example from a city in France will provide the open data other services it's neon and so I grab with WMS the imagery from from an area and yeah, it's The coordinate from this area. So, you know, this is a Robocat Pink tools command and With the only two commands we can bring the imagery and add them a tool to Visualize it with it. We do the same with lab with the labels And so with the labels we grab Jusison with a roof print from from the very same Area and rasterize them to convert from a vector to roster kind of data and I think we do that we can split the data between a training and the validation And here there's a few command that to help to split and if we launch first training We get as an accuracy Something after only 10 epochs Only on the building classification was already like this kind of result. So it's already quite good, but it's We'll see how to improve it and to improve it the point is to remove all the Labels were not accurate enough to be used as a training in the training that I said So if we look at the very first predict Result, it's as simple as to launch on the Train model with the image to to create new mask in the output how to detect Some prediction are or not Good enough is to launch a compared command between what We are supposed to to get and what is created in the output and we Use a true different kind of tree soul. The first one is related to the quality we want to Get and the other is related to Omnouch Feature are supposed to be in in this style. So I think there is enough Feature considered in a tile and if the quality is below a tree soul there is some Some pink area were spot on the map and the point is to to see that there is Wood area where It could be either No feature to to class to classify or good quality and there is some other parts Well, obviously there is some issue. So it's the first way to focus on places Who could imply that the human had to check and if we zoom in a bit and and more Indeed in the places where some square are spotted It would mean that for instance the label are missed in the input data and that there is something to To do just to explain you in pink. It's what is Predicted by the model. So it's from the imagery. What what we predict in in green It's what is supposed to be from the label So it's from the judge is son File we grab and in gray. It's when both are agreed So it's mean that both a prediction and the label Agree on these areas and pixels. So yeah, we can spot at once that these These buildings were supposed to be on the ground Related to the imagery are not present in the label. So what we want Here is to remove them because of the core of this Of the stuff is to say that if We've got a garbage in we will have a garbage out So we want to remove all the labels were not good enough to To be used and so there is another tool Still compare, but it's a side-by-side Compare with the imagery and with the results and here in this example there is a some some building stuff and the level is not at this moment and Reliable on the on the imagery so on this on this on this really want on the really want because we selected with a Lower quality of a result we can buy a Human operation remove it and once we remove all the old all those are well not Good enough we can remove them and lunch again the training and if we lunch again with an appropriate training dataset the quality of the results rise up and obviously if we Train it longer the quality will still Rise And will be better. So it's the first one way to Save your time either to grab them data to filter it as since you are able to To get good results for train model and if we look At the output once we train it We can compare side-by-side that's a for instance both In line so the imagery is In line with both the prediction and the labels or there that there is obviously some some change Some changes between the imagery and the labels for instance this part of the roof We are not present on the initial level. So it's a change all the it could be also false prediction In this case, it's a false negative because this one should be in a building and The point there and once we are able to do that between The open data and the imagery is to say okay What could we do now with the open street map data? So we grab open street map We extract them from the PDF to a jason Rasterize it on the recent ways and before and we can compare the results from the model we train before between Open street map data and get this kind of output with for instance here buildings were not Present on open street map as buildings. So we'll see I'm just just after what it could mean and Once we do that we are also the ability to vectorize it and to produce a new jason and to do everything we want with that so what What could mean the different kind of color if it's a green only? in this case it could mean that It's present on on open street map, but there is nothing on the imagery Related so it could be either that the building was built since the imagery or that the building was destroyed But still present on the open switch map or it's the open switch matter artifact if it's only in pink It could be in that It mean that it's predict by imagery, but not in open street map. So the first Frequent use case is that some as a political Gun is present in open open not Let but not with the attribute as a building. So it's it's only an area It's only a surface. So it's an attribute Issue But it could also mean that the building is really missing in open street map all that the building was destroyed Since the imagery or it could be also an artifact from the model prediction. Obviously not in this case and the third kind of Thing to to take in in consideration is also that It's not big. It's not because you train it with some building level that's the open street map level buildings are Always the same for instance here The parking are considered in open street map as buildings. They are not considered in the open data From Garnier and we took already as a building. So we have to to be To be aware that we have to deal with Closer Attribute classification Between the data we took as training and the one we want to compare with If we look about the performances For the whole data preparation it's something it's it's about one hour or two If we look about the filtering part It's some it's a human part. So it depends on you, but it's something Who could be down in a small amount of time something like one or two hours? the training part is a is The part of the process we consume the most of the Of the time of the process so with GPU it's something like a 20 minute per epoch so if if we want to training Till us till 100 epochs it will be more than more than one day And about the prediction We are able only with a single GPU to perform up to 3 megapixel per second So it's it's quite decent and so the whole process Will took something like two days between the download and all the prediction finished on the first time And yes, one of the point is if you have several GPU It scales so you could save time on the sport on this part About the stack so robot has pink is on the Reuse a lot of other existing software obviously And there are several stacks indeed involved first one here is related to Machine learning with a bytorch and deep learning the other one here is related to imagery with a pillow and open CV This one is related to the ability to to deal with styles And here it's the show special stacks About the timeline The initial project was provided by Daniel from Initially from mad box and the first release was Robo set There is several release of from Robo set till the point that Daniel left my books and so Robo set was no longer Maintain and so it begins to be robust at pink to to took the next step and to reuse several feature was initially bring to it Robo set and to add And to focus in fact more with the quality of data rather than the extraction till now What would be the next step a next step will be the ability to deal also with lower imagery resolution? So now we use quite quite good imagery to perform this kind of Prediction But it will be more interesting to be able to do so with imagery as something to to keep improving The prediction of performances. It's already good, but we bet we can do a far far more better and Next step will be also to to provide a pattern model to To allow you to directly predict and to not have to train Again a model from a net net but to to to directly a pre-trained model Maybe to to train it a bit with your really kind of imagery, but to save time on them on the training part I'll take away so Right now you can have state of thoughts Semantic segmentation ecosystem Available and playful it's on these two common lines You can use a plain open data to train a model and There the next step will be to to improve the Predict speed performances to be able to scale at large because here we are talking about a big city. We are not talking about a Country or continent or the world So if we want to to scale at large we bet that this this Dispatch would mean And that's it. Thanks Questions from the audience. I see one there Would you sort of speak up a little for the whiteboard? Sorry about that Regarding the performance Yeah What number of tiles are they? On the manual filtering and we so on the on the the training it's something like 2000 tiles each of 50 12 five five hundred and 12 pixel Square and About the manual filtering I grow something like 100 it's 1000 tiles and I remove something else about them. Yeah Yeah Anybody else how specific is the model it builds to a particular region of the world? The model on the train is our first specific to the kind of imagery we grab So if you grab a different kind of imagery with a different Radium Radiometric calibration your Your model will not be that good first and if you you change your your landscape Indeed it will not be accurate either. So If you want a model able to scale you have to cherry-peak a lot of different kind of landscape To do something able to to be representative to To what you are supposed to to classify and it's a part of the process to be sure that you get enough Of each different kind of landscape to train your model. Yeah Features Yeah right now On robust at pink There is already and the ability to extract roads from from the PDF. So that the first part and as since you get Since you get to labels here you can specify what you want So even if right now you can already grab easily Buildings and roads if you have already other kind of feature you want to classify It's some it's up to you The only point is that some what you what you want to classify should not be To sparse on the on the landscape. So for building on roads It's not an issue But if you want to classify something really specific It could be a bit a bit tricky If if your model is is trained to say always know It will be difficult for him to to say yes when you have to so I think it's not to sparse it will be fine Yeah, the point on The point here to to be able to to use it with With lower resolution I will be on two On two parts first one is with something else too We have a multi-bond Data and so right now Or this I think is able to deal with multi-bond imagery for training but not yet for prediction So it's something Must be improved and the other party is a resolution. So Yeah, we will play with Alphameter resolution to pixel and here we are talking about 10 meter per pixel. So obviously Obviously, we perfectly know we can't perform feature extraction from low resolution, but we We want to be able to to perform quality of data And so to spot if in a place For instance building a missing or road are missing. So not to to perform a feature extraction level But able but to be able to say that here The issue something missing or something is not accurate enough. And so a human have to have to check it again One key will be and the ability to to use a super pixel Solution who can help to to increase the quality of An imagery when you zoom in it's a kind of interpolation, but with a better output Resolution Um So the point is and as since you're we are using neural networks They are able to to learn a pattern from the from the imagery as since you provide enough information so for instance One thing a bit tricky with a classical classification system is how to handle shadows And here we don't have to buzzer with that because we provide enough information and labels and so The neural network is able to deal with Okay, I was asking about the image data sources Yeah, how to deal with that because I think that can be difficult From which satellites to get your images from You get it often In this example, I use open data Because it's available and because on this case They are quite recent if we are talking about something else too. It's a it's a weak Update, so I think there is no clouds. You can you can you can pick Your own your time frame as you want or as you need Do you see Does it change something to you is it something who could open some new usage from from or not or not yet Thank you Do you think it's got potential to be better at humans than spotting things Have spotting things in low quality imagery It is designed for yeah, it's the first design is that and as since we got A good quality on the prediction we can perform a feature extraction But the first point as you said both is to help to save time on the training data set Qualification and then to qualify The data set to something else. So yes, the point is to spot where a human Have to give a look and so to say this time Then there are many cases when you find ways Which you haven't found on the website Yeah, to shrink the hope is the last recent time this happened to me was Also, and it was in the world Mm-hmm Okay The point is really to first to to think about sparsity and the second is Do you have Labels accurate enough. Yes or no. I think so you These two points are Okay It's lead to to play more on your use case the first point Is it's a to sparse it really depends on which which are you take you take in charge If you you took only specific area related to grave To symmetry It could be not that sparse, but you will need an eye resolution and Imagery and you will need DT and labelled Labels accurate enough to train a model. So I think you you get that you can You can play with but if you only have a plain imagery and took the wall as a room the whole city area and one to spot grave It will be difficult Yeah, because because it will be a trend addiction. It will be something like that. So yes, you can you can sort it Okay Okay, so thanks, thank you very much I I Can keep my phone here so I can Something I have to work I Let me see if this works first. Oh There you go, I think it's my name For some reason it's online like I just need a internet access So basically I just need to go Oh this So I have two more projects to work together. I'm going to get nice and take a peek out of the room to do things. Obviously, it's a few specific places, but it's a really cool room. I'm going to get nice and nice and take a peek out of the room to do things.