 So our second talk today is shooting lasers at the countryside for faster internet So I'm James Harrison This is a talk on mobile mapping, which is a exciting and reasonably new field Which is starting to become accessible almost So I work for a company called giga clear previously worked at BBC R&D. We're hacking on archives Other things I do I look at stars. I occasionally make things and Partly build robots and then get canceled. Well, Roblox gets canceled. I don't actually end up finishing them Obviously I am not an expert on any of these things I'm about to talk to you about but I've broken enough stuff that I feel like I should probably come and share some stories and Yeah So bit of background on the company I work for because it's relevant because most of this stuff has been possible because of what they They do what we want is to give people faster internet in the countryside We build five to the home networks So we build fiber paths new infrastructure to homes in rural areas of the country And that means we've got to get fiber from a distribution point to properties over a very large amount of ground We've built around 70,000 properties so far. We're doing another 300,000 or so. We are hiring like mad So please come talk to your interested. We're in Abingdon in Oxfordshire So we basically did this sort of thing we drive around the countryside with more plows and that sort of thing And we're really interested in where we can do soft dig because doing this is really cheap and fast And if you need to start digging up concrete and things and that's really expensive and we try and avoid that so There's a pretty straightforward algorithm for this we want to figure out where people want fast internet and We then figure out how much it'll cost to go and build those areas and then we go build it simple, right? So the first bit is pretty easy modeling where people want fast internet. It's pretty straightforward You there's lots of no data sets and you can kind of do that Building it's also really hard because it turns out that digging up 10,000 kilometers of countryside is actually quite difficult Who's the thought? But the bit in the middle where we figure out how much it would cost to go build these areas and whether or not We can actually go and build those areas at all. That's really interesting because if you get it wrong That's a big deal But if you get it right that lets you go and build areas that people didn't previously think were viable and if you model that Well, it's great. And the trouble with this is the existing rural mapping sucks This isn't the digger OS OS are fantastic Master map is the commercially available best map of the UK bits of it are starting to become open days in now So if you're interested in this go look at what surveys website They've got a great map. It's the whole country in and it's a cadastral map, which means it has no gaps It's got a complete set of polygons that cover the entire surface of the UK. This is brilliant But it has a code of roadside natural which describes Stuff that's by the road and that is of a natural construction It's not man-made and that covers a lot of things like verge Which we can dig in easy Very thick hedge, which is really quite tricky We've had it miss things like streams and drainage ditches and sometimes we can get drystone walls where people have built Walls not told the OS OS haven't resurveyed the area in years and We go and try and dig holes in there and realize there's a wall there and we've had houses and things So this is the sort of thing we've sometimes found there's a hedge there. It's quite thick. This is a point cloud We'll come to this in due course And again something where it was just missed out that there's actually a house built on the side of the road here In the maps this looks like a continuous piece of verge that we could just plow through Not gonna happen. Unfortunately These sorts of things are significant because they do eventually cost us quite a bit of money And that can really affect whether or not we can go build in an area so we send people out to go and survey an area and Turns out that takes a lot of time and lots of people So if you're building fiber to the home in urban areas like BT and city fiber and other people You're averaging about 10 meters between properties. We average 80 meters so Straight out the gate. There's a lot more stuff to do Trying to get all that data back You can send people out to go and look at things and try and understand the terrain and whether there are any Obstacles and so on but getting all that information back in a way It's useful that lets us then correct those problems is really hard and trying to manage that much information at scale turns out It's You don't want to be doing it if you can help it Satellites are not useful for us because they can't see through trees and lots of our roots are covered in trees And they don't have great resolution Not quite sure what's going on with the display, but No, that's not happy at all is it No, so while we're trying to get this sorted Aerial photography is another thing we've looked at doing and again has the same issues with tree cover You can get a bleak imagery, which is where someone's taken a picture from the side And is looking at things sort of edge on and that can help but in most areas in rural areas It doesn't help that much at all See if we can get this working again Hey, okay, and Google Street View is another option that everyone goes Oh, she's street view and go through it and look at all the pictures and street view most of the countryside Google didn't resurvey since their original survey. So six seven years old is normal In rural areas, which isn't very useful for us. So mobile mapping is whether we come to Basically it's a process for capturing geospatial data from a mobile vehicle With a bunch of sensors strapped to it normally including cameras sometimes including things like light up So in order to do that, you need some sensors You've got to be able to figure out where they are. You've got to build that all into a vehicle you've got to get all the information somewhere usable and Somewhere accessible and make some tools to work with that data so made the main sensors that we're talking about today are lighter, which is a form of laser scanning and Cameras cameras are not useful for spatial mapping You can do photogrammetry, but it's really intensive CPU wise So it takes a long time if you've got a lot of images to get high quality photogrammetry And it's also quite prone to error So we have a classic case where we drive along and we take lots of photos of verge We think it's verge. It's actually a bank of 45 degrees, which is a real pain for us because we can't plow in that effectively So you try and Figure that out from photogrammetry. It doesn't always catch it So there's a few things where actually doing direct observation of distances is useful for us But cameras are really good for context So laser scanners Laser scanners are of two main types pulse based and phase based pulse based ones are usually cheaper and Usually with a pretty good Spatial resolution accuracy and so on phase based ones are more accurate, but have a shorter range typically So you it's a pretty simple idea you fire a pulse of light You wait and see for any reflections come back at that particular wavelength. You've just shot pulse out at and you then Have a think about it figure out where all those things that you just saw were in space based on time And so the nice thing is you can actually do Detection of multiple objects with good enough scanners that if you fire a pulse of light Say at a tree then the light might pass through a Leaf and hit a branch behind it and you'll actually see a reflection from the leaf And you'll see a bigger reflection from the branch which stops the Pulse of light and this works really well for mapping Verges and things because it means you can actually shoot through things like hedges and typically catch enough of the surface behind that Hedge that you can actually measure what the verge looks like below the vegetation And you can then figure out what the vegetation looks like how much of it there is and so on So scanner I'm most familiar with and that we use is a scanner from a company called Regal It's pulse based it can fire 1 million points per second and it has a mirror in the back of the scanner Which can spin at 250? Revolutions per second so 15,000 rpm. So that spins around and we effectively drive along With this on the back and it spins around and each time it spins. It's firing out a Large number of pulses which we're then capturing information from so this gets you initially some Events that are in the scanner's own coordinate system. So basically the angle it saw is at and an X and Y offset and Once you georeference that you get a point cloud point clouds are literally just a cloud of single points in space and That gets some if each of those points will have some attributes with it Like whether or not you saw multiple echoes or maybe it's just the point itself was a first echo last echo And if we want to we can add extra information in there We don't get any color because we're shooting in infrared. So effectively what we get is a an intensity view of the world in infrared but we can then overlay that with images to make some colored points and Usually we end up with horrible vendor specific formats, which we know to work around which is no fun so in theory We've got a demo here, which might or might not work Okay, I'm gonna go with kind of so this is actually a point cloud of the campsite which I can't see to control so this will be interesting and so this is actually a complete scan of eastern the campsite and grounds This has been georeferenced and colored. So we've got lots of metadata information about it So this is the sort of output we get this is online by the way if you just Look me up on Twitter. I'm at James Harrison. You can find a link to this. You can download the whole data set for free So thank you for my company for letting me drive the vehicle around here and collect all this So this data set is the output of About half an hour driving around site and then about two weeks of fiddling around processing it It's not usually that long, but I don't usually do the stuff hands-on so Normally it takes us anywhere between three to five days of processing to get a decent output And the nice thing about point cloud is that it's accurate within its self as long as your georeferencing is reasonably good So within a particular scan line, you might be talking about two to three millimeters of accuracy Which when you think you're driving along and doing a million of those measurements per second It's pretty impressive And you can take angular measurements you can take distance measurements using whatever tools you like and figure out lots about the world without having to do a huge amount of work So all of these things are pretty safe if you start doing aerial lidar that changes Most aerial scanners will be class 4 and we'll burn your eyes out if you look at them on the ground But all of the stuff we do on the ground is very safe low power nanosecond pulse lengths It's all pretty good Other stuff that you can get and if you're looking for a lidar sensor to hack on velodyne are usually the go-to answer They have a range of quite cheap sensors around the sort of I say cheap. They're still around two to three thousand pounds at the moment That's cheap in lidar terms and That is about as cheap as it gets today Z for last plus F are Probably the gold standard those are phase based sensors and that's pretty much the best you can get those are very very expensive Six figures and up quite happily We also want to take some pictures and if you're driving along at speed and you want to take good pictures Then you need to consider a couple of things you want a lot of dynamic range in your images So 10 bit or 12 bit dynamic range It's like you basically drive along and you imagine if you've got trees on one side of you and you've got bright sunshine on the other You can still see detail in all of the trees down here, but you can still see the detail in the bright verge to your right And we do some post-processing stones a bit so we can consume it easily Because we're driving along and mobile mapping rigs. We want to be driving at carrying twice speeds So we're trying to do You know 30 to 70 miles an hour We don't want anything it's going to give us lots of tearing in the image So we need a sensor that rather than being a rolling shutter where the image is read off from top to bottom Progressively is a global shutter. So we take the whole image data at once so The other consideration is whether we're using planar cameras or panoramic Plane of cameras usually get used on mobile mapping rigs as a way of adding extra information to panoramic imagery So you might have a detail camera looking at a road surface particularly hanging at the back of the vehicle Panoramic cameras are still what you need if you want The good contextual images that let you kind of get that Google Street view feel which is a big thing for a usability perspective The state is really hard to use unless you kind of think through how you're going to access it So most mobile mapping rigs and ours included use a camera from FLIR called the ladybug 5 It's a very common panoramic camera 30 megapixels global shutter 10 bits of dynamic range and Please if you're ever doing anything with rugged connectors, never put micro USB 3 on it ever Never that's a weatherproof USB 3 connector. We've killed three so far. I think You don't want those. Please use 10 gig base T or something So we now need to figure out we've got our sensors We now have to figure out where they are at each moment in time to be able to place that information in context And to make it a lineup So we need to use a coordinate reference system in the UK We have any ordinance surveys national grid and that's got a good relation to the all of the geodesic system used by satellite system So we can refer to that quite readily and accurately and it doesn't move over time Whereas the if you try and reference things to latitude longitude due to continental rift All of your measurements will be wrong increasingly over time So about two and a half centimeters a year at the moment. So We use these things and all things exclusively in post-processing effectively So GNSS GPS is one GNSS system GLONASS by do got a layer of other GNSS systems Those are how we figure out where we are accurately used to be do triangulation and things but Nowadays, it's pretty much the gold standard for surveying is a GNSS survey done, right? And you can get pretty good accuracy without needing any external correction sources, but still only a meter or so if you want to get better than that then you've got to correct for a bunch of different errors mostly atmospheric errors and pretty much the only way to do that is Use a better antenna to get better signals clear and reduce the effect of things like multipath interference use more satellites, so you've got more data to work with and Use more information about what the satellites were doing while you're recording your position So actually having accurate information about where the satellites were calculated after the fact But the only real thing you can do to make huge difference is to use a base station to correct errors So something that sits stationary so anything you measure in terms of movement, you know is error simply We are kind of care about accuracy a lot in this because When we get to processing trying to make overlaps and things like that will make sense If we have a more accurate trajectory to start with then we have a much better chance of making everything line up at the end So we're trying to aim for centimeters not meters and that's not an easy feat You typically end up needing to put a base station down The ordinance survey have actually got a huge number of existing base stations that run all the time and you can get the data for and there are commercial services that will take that and Interpolate between multiple base stations to figure out a virtual base station near where you're doing your survey work Which is usually faster And that lets you effectively figure out what the error is at that particular moment being introduced by the atmosphere and so on And that means you can correct your position down from meters to centimeters Now this is all getting a bit expensive It's getting cheaper happily much as lidar. Hopefully we'll be soon. I'll talk more about that later, but Survey grade receivers are still in the range with 10 to 15 thousand pounds if you're looking at cheaper systems That aren't quite survey grade But are still corrected with RTK, which is that the way you transmit these connections Then you are still looking at maybe 2,500 pounds. There's a now I think will be emulated reach which is about 700 pounds a station So it's starting to come down in price a lot of the stuff that people have been doing with drones has helped that a lot Because suddenly you need really cheap positioning because you might crash the drone. You don't put 10,000 pounds of scanner on it You can go open tools like RTK Lib for correction, which is kind of handy And you can get the correction data from OS for free so Once you've got a rough idea of where you are in the world in absolute terms We start to figure out where we are in terms of our attitude of the vehicle. So as we drive around we're rolling. We're pitching we're Turning around and GNSS even with multiple antennas and receivers won't give us that dynamic View of the world to the accuracy that we need in terms of temporal accuracy and spatial accuracy So in order to figure out where we are Any particular moment in time bearing in mind that we're doing a million points per second 250 Scanlines per second. So we really do care about it being precisely timed as well as Precisely in space. We need an inertial system. So inertial navigation systems look like these exciting boxes here Usually are integrated with the GNSS receiver. So there's a box that does your positioning and you have an antenna attached to it So typically these are made with MEMS accelerometers and those are exactly the same sort you find in your phone They're slightly more precise versions of it, which justifies the higher price tag And the same goes to the gyroscopes more advanced survey systems will use fiber optic gyroscopes Fox Those are much more expensive. They use a Magic System they're complicated They are really expensive to make they're really really really really expensive You're still looking at 10 points around plus for each of these things You can still get pretty good performance out of cheap GNSS receivers and IM use It's good enough for doing a lot of things if you're not doing laser data You can use a lot of this stuff that's out there if you're doing laser data You still end up needing to stray into this sort of territory So these things will be accurate enough that you can know where you are absolutely in the world without GNSS for about a minute You might get five centimeter accuracy still, which is amazing We also use GNSS as a base reference so as a time reference So all of our sensor data is being captured and time-stamped all of our position data is being captured and time-stamped so we know any particular moment where we were and what we captured and All of the things like event triggers for cameras are on that common time base So we just need to get all of our GNSS together Get rid of anything that we had really bad GPS signal Stuff we might introduce error with We feed all of that data and the inertial data into a Kalman filter which is a way of integrating all that information and Estimating some of the internal state of the the inertial system so things like gyro bias and things that let us figure out a better solution and We run that through a few times and we get some accurate position data around and We're typically to help that process. We actually drive for maybe half an hour before we start capturing any data at all Preferably on some nice fast roads with some slow corners in and that gets us a pretty good view of what the INS is up to before we start capturing data So what we end up from the GNSS is something it looks a little like this So this is just in green or through to blue Estimate of the position accuracy we're getting from GPS. This is the eastern side So we had pretty good GNSS throughout the entire survey and as we were driving away We kept running the recording there and you can see that we had some dropouts We had some periods where it was particularly poor and those areas We typically would either exclude or just treat less trustfully in the the algorithms that are processing the the data One other thing you can see there a green triangle that green triangle is the virtual reference station that we used to correct the data set So it was calculated So typically once you've done all of this stuff with all of this fancy hardware You can get down to a positioning error in absolute terms that looks pretty good and we're saying there that we're Usually for XY positioning below about three centimeters accuracy and we've got that at 480 Hertz Temporal accuracy so that gets it gets us a lot of data to work with We're trying to get the scan overlaid on top of our existing maps because we want to use them for Enhancing our existing maps. So we're usually targeting sub decimeter accuracy So below 10 centimeters in absolute terms the national grid So I could tell you whether or not it you're in one of those tiles on the floor or the next one over or better trajectory processing so figuring out that trajectory and making it as accurate as possible as we spend a lot of time Manually to try and get it as good as possible a solution because it's going to cause us loads of issues later on in processing If we have a really bad trajectory at any point, so we're really keen to get that as good as we can So once we've done that we might have some overlapping flight lines This is again the eastern data and you can see in different colors there different bits of the trajectory that we're processing separately and Those are overlaps because we've driven the same route more than once and you can see On this diagram You can see over here. We've actually got multiple Posts appearing and actually there's only one post we're driving past that the row of posts just up there and Because we've got multiple overlapping scans that have got slightly different positioning We're recording that more than once effectively So if we don't correct for that then we end up with interesting side effects like this so this is where we've got some Multiple objects appearing and again you can see there's actually a lot of error here And this is something due to the process of the data incorrectly If you have angular errors, then you get slightly different sorts of weirdness appearing so the trouble with all of this is that Everything literally everything in there is proprietary Standards proprietary formats and we have to get somewhere useful Which means we want to get outside of those preventive proprietary formats and into open formats So we process our information We do some basic classification of those points so we can say that anything's directly below where the scanner was was probably ground And then work our way out from that and try and label all the points to say okay, this is ground This is some vegetation. This is maybe a building looking at things like reflectance of the points We get rid of things like noise. So we sometimes while we're driving along we might scan Rain we try not to birds Or Occasionally you drive through a scan and you see this weird bird shape in the middle of space It's just we happen to be driving past a bird that was flying past us So we try and go through the data set and remove all of those sort of points because we're not actually interested in whether or not There was a bird there kind of cool, but and We then try and over align all those overlapping passes and get everything snapped to a single surface and Corrected for x y error an angle angular errors and so on you're solving for sort of nine parameters For each bit of trajectory. It's a very complicated thing And it doesn't always work perfectly And then we try and color the points in by taking our images overlaying on top of those Point clouds our images and positions and then casting rays out from the images the images are much higher resolution than the point clouds So usually we've got a lot of pixels to work with so we'll take an average from the nearest three pictures and the sort of thing So Unfortunately the hardware still to is inaccessible for this to have really become an open-source thing Hopefully this is getting it's going to become more accessible because the cost of the sensors is coming down and More data is becoming available online. There's a great set of data for aerial stuff at open topography dog And lots of information you can find there I'm gonna start speeding now because I've realized I've only got Yeah, not that long left three minutes left, okay So there's lots of potential for new algorithms and stuff we haven't really worked with yet We generate loads of data about a terabyte a day and we store all of that and then we do it on an s3 And use AWS for a lot of the processing because it's cheap and easy and it scales pretty well for this So we're capturing roughly five fifty thousand points and maybe two to three billion points of data per day of Information about the world we will get rid of some of the photos and things because we don't need all of them But we typically keep the point cloud for life We use QGIS a lot, which is our open-source GIS system and we built a plug-in which lets us access this information through all of that and some Custom rendering stuff, which I'm hoping will open source at some point soon So we didn't build our own one and we bought one off the shelf from a company called 3d laser mapping there are lots of other companies that do them and So we ended up with a van that looks like this. It's bright orange because that's our thing and In the back of the van we have our system, which looks like this You can see the laser scanner there the cameras on top the GNSS antenna on the top of that It's on the left So we just raise it out at the back of the van when we're at the start processing and off we go Other systems out there things like this is the Leica Pegasus 2 and this is trim balls and x9 It's got two of our scanners At different angles so you can record more points so It's kind of this is the sort of stuff you get out of it. This is from kind of over there And this is the intensity gradient. So this is showing you differences in reflectiveness between Different points so you can see here where we were driving over Was actually some tracks and so those will reflect more brightly in infrared. So we'll get a higher return value Interestingly with things like trees and so on branches will typically reflect more more brightly than vegetation Well, so you can actually take pretty good structural images of trees to figure out how many branches There are all this sort of thing or to see what sort of branches are there and you can things like removing vegetation to Virtually strip a tree back. She's interesting So the RGB stuff mostly works you can see some up at the top here some white pixels here We couldn't find any colors were blind. So we just guessed and in terms of detail There's a wire link fence somewhere down there And you can see you can pick up all of these individual wires. No problem at all With a very fast scanner, you can pick up an amazing amount of detail more Exciting pictures of things being not properly aligned. So You can do some interesting things like with this like you can take height maps and use those for Figuring out where you should put things in the world But there's lots of things you can do with this data. It's really really powerful There are all sorts of errors that you can introduce by processing incorrectly So here's a slight issue where these are not obviously natural features We've got a couple of flight lines down here that are perfectly fine But one that isn't because we've not managed to get a good correction solution for that. So in that case we've Will probably usually will just delete that flight line because we've got a good ones that cover the same area Some things we've hit on the road You should always check to make sure that roads are not private roads We were warned off this particular road by a gentleman with a shotgun We did not hang around And you should always check the height of bridges before driving under them so Very quickly so I'm being hurried on If you want to do this yourself, you can do a budget version Panoramic cameras and are getting pretty cheap laser scanners are getting cheaper Hopefully the stuff that's happening with automotive automation and Self-driving cars is starting to push push real change towards solid-state sensors, which will be drastically cheaper Than the big expensive things we have now and you can get pretty good I am used and GNSS for not a stupid amount of money still expensive, but it's getting there and If you don't need perfect precision and perfect accuracy You can get a long way with what you can get off the shelf for not a lot of money today Today if you want to go build what we did or do those sorts of things You're looking at half a million pounds plus if you want to do it on the cheap You can probably do it for more like two to five thousand pounds Hopefully in the year or three's time you may be looking at five hundred pounds. This is awesome because it's a really good way to collect data about the world at speed and to Try and take some power back from companies like Google Apple who are the only people who can go out and map these things at scale at the moment Then actually put it in that data in the hands of more people If you want to play around with the point cloud data Then there's a link in a second, but there's some great tools in PDAL Python and one cloud library Those are if you've got those three tools you can do a huge amount And yeah, there's a link there for the data if you're interested Come ask me questions and huge shout out to Andrew Godwin who made a complete 3d print of the campsite which if you want to come and look it up here