 So, hello everyone. Sorry for my voice. I am Ilya Zverev from Maps.me and today I will talk about pain and disappointment also known as OpenStreetMap. So, OpenStreetMap is really the best map there is. I think most of you know it. It is very big, it has striving and amazing community. You can find anything on the map from places and borders to roads to street lamps and manholes. So, if you need anything just go to OpenStreetMap and you will find it there. And a lot of people are using OpenStreetMap now and a lot more are thinking of using it. And the thing is when you are using OpenStreetMap for a long time, soon you find yourself regretting that you are using it. Because from a distance or some seems like a perfect map, but when you use it you see that it has a lot of flaws. Some small, some big. I can name just 100. So, it got awful data model. The tagging is inconsistent and the quality varies from city to city. Your village may be mapped perfectly. The next one is just a couple of roads and so on. So, the reason for all of this is because OpenStreetMap is made by mappers and four mappers. And everything OpenStreetMap is is decided by mappers. So, the data model is very simple when you need to learn it into map. Just points, lines, tags and attributes of features are also very easy. You just upload some random strings and the feature is present on the map. But when you then need to use the map, it turns out that what is easy for mapping is not as easy for using the map. As I said, the data format is awful and the OpenStreetMap is always broken because from simple concepts, mappers tend to build very complex abstractions, very complex tagging schemes like road lanes or public transport or route relations or anything actually. Yeah, 3D building is excellent example. Yeah, so this is my story about one of these complex abstractions that I tried to pull from OpenStreetMap. It's about subways. So, I work in MapsMe, which is a cool offline application that uses OpenStreetMap. And once every couple of years we leave our cities, leave our families for a couple of days to eat meat steaks, to ride bicycles and to code days and nights long. It's kind of a small hackathon. So, this year there was a small team with which I was working. We thought it would be fun to add subway maps into MapsMe. Just offline maps with routing, with, I don't know, map. There were people who worked on routing, on design, on rendering. I was working on data. So, and pulling subway data from OpenStreetMap is quite easy. You get objects like a station equal subway and the subway station, great. And you download route relations, which order the station from first one to last. So, you have stops, you have the coordinates, for interchanges you can just find stations that are really close to each other. So, in a couple of days I made a script that downloaded this from OpenStreetMap, converted it to a simple format of lines and points. And for Moscow it worked pretty well. In the next day I used my free time to process some other cities like Seoul, Barcelona, Minsk and so on. And we got pretty fine rendering all of that. And I overheard that our CEO said that if he likes the result, then it will be included into MapsMe itself. And I thought we shouldn't allow that. So, on the final day when we had presentations from each of the groups, I especially spent, yeah, the screen was very strange. I specifically spent five minutes talking about why pulling subway from OpenStreetMap is especially a bad idea. And nobody should do it, because the data in OpenStreetMap is always broken, because it's relations and nobody knows how to use this. And, yeah, so you shouldn't. So the hackathon ended, we all went home, and in a month our product manager comes to me and says, well, we're doing subways. Oh, shit. What could I answer? Okay, doing subways. So to pull subways from OpenStreetMap to use all this mess, I needed a plan. This was my plan. Then I thought, well, I need to write validator. What is a validator? Validator is a website that's regularly updated that downloads OpenStreetMap data and checks it for correctness. We have a lot of validators in OpenStreetMap. It's administrative borders, checks that they're not broken. It's places like citizen town, it's roads, it's everything, actually. So I need to make validator that checks that subways are mapped correctly. And what do we start with? Well, first, I need to see where are the subways, where to look for these relations. So I open Wikipedia, obviously. And Wikipedia has a lot of lists. Of course, there was a list of metro systems. And I clicked on every metro system there is in Wikipedia and found out there are 180 systems in the world. Subways, light trails, urban trains and so on. So I prepared a spreadsheet with city, number of lines, number of stations and bounding blocks. Pretty handy spreadsheet for each of these networks. And now the next task. How to get the data from OpenStreetMap? Well, the chart I showed before with stations and lines, that is a very simple option. And we don't use this. OpenStreetMap is not simple. So this is more complex, not the full schema. But this is a subway station. So the subway station, as part of the route, contains railway tracks for both sides. Platform, obviously. Stop position, the train stops. And a point for station itself. It's not a physical point, but it has all the data on the station, like its name. And also there are entrances where you should enter to go to the station. And there are subway routes, which include some of these. So the first thing is, these are all disconnected, separate objects. So how do you know from a stop that it belongs to a station? Or more importantly, from entrances? So all of these are joined with a stop area relation. Relations in OpenStreetMap just group some things. And then interchanges. So when two stations are close to each other, you can go through a tunnel from one to another. Then there are two stop areas, like I said. And these are grouped also in a bigger relation. Relation of relations, you know, OpenStreetMap has simple data models, but very hard to use. And this is stop area group relation. But wait, there was no stop area group relation. So you know, OpenStreetMap has public transport mapping schema. It has two versions, actually. There was version one, version two. And turns out there was no way of mapping subway interchanges. So in nine years people were mapping subway in OpenStreetMap. No one has thought of mapping whether you can switch from one station to another. Which shows that nobody even tried to use it for routing because it's important information. So how do we add stop area groups? Well, you make a proposal. Taking proposals is a fine topic for ten other talks. But basically you write that I propose to add this type of relation and so on and so on. So I wrote a very big wiki page, obvious error from my side, on taking subway stations in lines. Everything from station node entrances to railway tracks to stop areas to stop area groups and so on and so on. So a person who visits it and read it immediately knows which text to use. And it proposes some changes, obviously. So I wrote all that and published it so people could discuss it. And there was a very big discussion over several mailing lists and chats and so on. The discussion page in wiki was around 50 kilobytes. Well, that's an awesome community. You get used to it. So, and I learned a lot from the discussion. I was constantly changing the proposal to accommodate things. For example, I learned that there are three types of interchanges, not one. One interchange when you just exit the train, don't go anywhere, wait for next train for different lines. And interchanges when you have to go to the tunnel to change platform and use it. Or when you have to leave one station, go to the surface and enter another. And so on. These and previous pictures are all from the proposal. I tried to make it as simple as possible. So, meanwhile I was working on the validator, as I said before. And it was coming along nicely. So, the thing I did was download the planet, you know, 38 gigabytes of compressed data. Filter it out so it has only subrelated data, half a gigabyte. And read it, sort features by bounding boxes from the spreadsheet. And it prepares this structure in the memory. So, for each network there are a number of routes. Roots contain several variants, like forward to backward direction. There are tracks, stops. Each stops basically contains coordinates and stop area. You know what stop area is, I hope. Stop area has stations with all information for station. And of course there are transfers. And I ran this validator against the planet. So, how many metro networks were mapped reasonably well? Well, not much. Yeah, of 180. So, and the release was planned like in two months. So, how do we speed this up? So, what did we do? We hired students. Just a couple. I wrote a long tutorial on how to map subway stations. And meanwhile I also sat behind the computer. And to start fixing some bugs myself. Yeah, my computer is very slow. Sorry. Top of the line MacBook. And it can show animated gif. Just a moment. Okay, so in a month we have 40 networks. And awesome. It looks like we're progressing fine. But at that time I felt like I'm failing on all fronts. And why is that? Because the proposal was put to voting and it failed. Because there are many mappers in opposite traffic. You know, it's got a community of 100,000. And everyone had an opinion on the proposal. What to improve, what to change. Some people didn't like color. Some people didn't like, I don't know, nose versus waist and so on. So it got rejected. Well, my fault it was too big. Students also weren't worked as much as I hoped. They were making mistakes. They didn't finish their job. Well, I'm not good as a mentor, but still. So what did I do? I went mapping myself. For three weeks I was fixing and fixing errors I was processing. Networks in cities like London, Berlin, Paris and so on and so on. And finally in December we released MapsMe. With many subway networks, almost every network in Europe. And some in Iran, Japan and so on. So yeah, that's the win. It looks quite pretty from last week to October in Brussels. So you can plot your route to central city from here for example. Well, you can't since there are no subway stations nearby. And yeah, it's pretty cool. As for proposal, I've currently made a second try with much smaller, much simpler proposal. I hope it will work. And as of this week, we've got 120 good networks. And why is that? It's not that I went to fix all of this. It just, we published MapsMe and people from the three maps started using the validator and started fixing errors themselves. So we've got almost all of South and North America covered, including USA. We've got 60 networks in Europe ready to use. But of course, this won't last. And as I said before, opposite map is always broken. If unattended, all relations will perish and be obsolete. And that's why we need you to help with this. And what can you do? Well, you see the website link. First, of course, you can map. Not only open JOSM editor and fix some errors from validator, but also if you know that in some subway network there are new stations opened or closed or something changed, then you should go to the spreadsheet and update numbers so we know. For map or snow, something has to be updated. And second, most of you here are developers. So please, all of the things I am talking about are open source. The validator processor are published on GitHub. You can download a JSON file with all the networks that's generated twice a day and use it in your application. By making a map, you can do offline routing with the data. You can write a new application that shows where can you get from here using subway networks. All of this is from OpenStreetMap, so it's open and usable. And what's for this year? Well, our project manager came to me a few weeks ago and said, well, great job on subways. How about, I don't know, trends, maybe? So this year, I guess, we'll have even more data from OpenStreetMap and hopefully not as broken as it was before. So thank you. We have time of like five minutes for questions. OpenStreetMap data is that it only works if you use it. I mean, if somebody just adds some data and then forgets about it and you call it garbage, it's only when there's a group of people who use it every day that, oh, this is stupid, we fix it. So I think before you go out and fix all the data, make the applications and they'll tell people, OK, start using it, it's not broken, and then get people to fix it. Because otherwise, it just doesn't work. So your people are crazy enough to really go out for all the time, tell your business and compare it with what is in OpenStreetMap. So this number is wrong now. It's not how people work. You're correct, but this is a bit of chicken and egg problem because MapsMap is used by 10 million people every month. So do you really want to take this broken data from OpenStreetMap and give it to 10 million people? They might be not as satisfied. No problem. This is the way you correct, right? Yeah, but you have to do that. Otherwise, you get to check in the next situation. OK. Are you the first one to use this data in navigation? Yes, I'm pretty sure I'm first to use any public transport data from OpenStreetMap, not just for highlighting roads that have buses and trains, but to use the data for navigation. Nowhere I just took some random numbers and produced some kind of timetable. But with subways, you can be pretty sure that there are trains every two to five minutes, so it's not an issue with subways. Of course, it will be an issue when we do trains, for example. But we'll think of something. Do we use other world data that formats me? Not actually. Tests will never be a strong side. After building offline data, we check that our car roads weren't broken and some other tests, not as comprehensive as subway tests for this project. Because most of the data can be used pretty easily. They don't require such testing. I think the question was, could we use GTFS data for this? Very few subways in the world have GTFS. Like, of 180, my best guess would be 20, 30. In most cases, GTFS is absent. It's a privilege for North America and Western Europe. We would use it if there was such data. But of course, we have plans to employ GTFS in the future for other types of public transport. We as a company don't, because we don't have a single map employed, but people, mappers from OppositeMap are free to do it. There are tags for ticket machines and for everything, actually. And whether we would use the data, I have no idea. If you submit many bug reports about this, then I think it might get in the road. Sorry, what is open? All the things that I was talking about is open. So the code is on GitHub, and the data it prepares so maps me is also published on the Internet. You can just look into it. Look how I make the fake time tables and distances. So it's all on GitHub. So if you can visit, sorry, it's very slow, pt.smz.ru, all the links are there. It's just a small website I hacked up last night, so it's not very fancy, but it has all the links. Okay, one more question. Well, can users change data with maps.me? We have an OSM editor inside maps.me, but it can be used only for adding new venues and changing tags on existing venues. As for subways, the data model is too complex to allow editing from a mobile device. I don't even recommend editing it with a web editor or an opposite web website. Jossum and only Jossum. So no, we won't allow editing subway data from an application. You have to go through a proper editor. Okay, yes. Thank you, everyone.