 Hi, I'm Sanjay Bhangar, I work out of Bombay. I work at an organization called Camp that three of us performed about five years ago. We're kind of an art space. The reason I'm showing you our website is for some reason. We had interns working with us for, this was about four, five years ago, and we weren't sure what work to give them. So they actually sat and took Google Maps, like screenshots, and put it together in Photoshop. And then we printed this kind of really large map of Bandra where we work. And that also ends up being the background to our website. This doesn't really have anything to do with the presentation. I'm going to show three kind of unconnected projects. The only kind of thing connecting them is that I've been involved with them one way or the other. Two of them, I haven't done the majority of the work on. So they're really complex questions. I'm probably not going to be able to answer and going to refer you to the developer's Twitter IDs. But I've worked quite a lot on these projects in one way or the other. So let me start with the first one. It's primarily a video project. It's a video archive project we've been working on with Jan Gerber and Sebastian Zuckert from Germany who've done most of the heavy lifting. So this is a website that they run and formed the inspiration for a project we started in India called Padma. Their website is called 0xdb, which is kind of a visualization of their downloaded movies. So it's kind of like they've downloaded like tens of thousands of movies over the years and kind of wanted to have some good way to kind of filter, sort, have people make lists, share those lists, et cetera. So one of the things that they really wanted to do is display the movies on a map. So there's several other components to the website as well, but I kind of focus on the maps. So this is a website that they started about six years ago. And then we started Padma, which looks very similar in terms of interface. But it's kind of working with documentary filmmakers, mostly from India. And the big difference is on 0xdb, all that data is gathered by crawling the internet. They pass IMDb for place names and for other things, et cetera. On Padma, we put in all the data ourselves. And on Padma, it's kind of documentary films that we are free to show the video material as well. So you can see the video material as well. On 0xdb, you can see all the metadata, but not the actual video material because that's a copyright issue. So what it went back to in the past three years is realizing that dealing with large data sets in general is a kind of interesting problem online. And when you're combining location data, other sorts of metadata, et cetera, then you really want some sort of library or framework to deal with this. So it's been spent building this library, which we call OXJS, which is a JavaScript framework to build web applications. That's kind of highly ambitious. If you see any of these websites, they're all one page. You load one page which basically has open body, closed body. That's it. And one script tag that then loads all the JavaScript, handles all the URL changes, et cetera, et cetera. It's a bit ambitious. I think it's something that we'll start performing really well probably in the next few years, but we thought it might be an interesting thing to do. So just to show you how the maps work on this front end is the movies have all been passed for location names and have been passed at the clip level. So the idea is that get subtitle files for the movies, run geolocation on them right now in a naive way of just having a list of existing location names that you search for in subtitles, and then geolocate those clips of the movies. So just really quickly let's click on South Asia or maybe zoom in. So I'll get through this map interface that you see that looks kind of weird because it's Google, but it doesn't really look like the Google Maps interface. So OXJS has a kind of wrapper over the Google Maps interface. And there's an open bug report. We swap out Google for open street maps and open layers or leaflet or something. But development on this was started about three or four years ago. So the choice was made to use Google, but it's a complete mapping library written over it. So I'll show you some of the other things that have been done with the mapping library that we're going to do with maps almost. So this is just to demo you how it works. The idea is that you can click on a location. This doesn't actually have clips. So if I click on Mumbai, it rolls through the archive, shows you all the clips that are tagged with that location. And then you can actually watch those tiny five second clip that's been tagged with the word Mumbai right there if our internet connection is decent enough. Otherwise you can try and then I can double click to see kind of the details of this video, et cetera. So this was kind of from automatic metadata. And then on Padma, we've got this kind of labor of love kind of like people going in and marking out clips of videos. So you can see there's a really heavy kind of density around Bombay because that's where a lot of our video material is. So I'm just going to kind of zoom in. So we're going to try and keep a track of time because I have to show three different things. But you get the idea that there's a lot of little locations in there. And this, again, has been done in a kind of like manual data like people put the names as an interface to kind of do a geolookup and then kind of mark out where that location actually is for some reason the server is sometimes standing out. But you click on a location, you get the videos, so I can also turn on stuff like these filters. So I want to see only the videos that have the topic leaks, for instance. We probably won't, but if we say cityscape. So you see all these filters kind of work together. So you can filter by different keywords. You can filter by locations on the map. You can filter by people, et cetera. So this is possible with the OXJS library, which is OXJS.org right now. So what's on the website is basically just a checkout of the code base. Because it's entirely client-side. There's nothing happening on the server. So you could run this entire website either on the client or the server. Just to show you some of the map examples really quickly, the document. So there's an OX.map. There's a few different OX modules as a UI framework, which has stuff like those infinite scrolling lists, the buttons, the menus, auto-completes, stuff that you would need for a UI framework. And it's got a geo and an image framework. This is something, again, I don't have enough time to get into the details. Let me just show you some demos of stuff that's been created with it. So this is also a kind of nice interface to browse the code. You can just see the code there. You can see some comments and documentation. So again, you guys can go over this stuff. It's kind of well annotated. And then you can see this slide. So here, what this is doing is it's taking a list of cities from geonames.org, parsing the JSON structure, and rendering it on a map with the kind of list filter. The actual code required to do this after using the library is actually quite minimal. Because there's a thing that generates your list for you. There's a thing that parses the JSON for you, and then throws it up on a map. And you can double click to zoom and get data about the area. Just some fun things. One of my, this is pretty cool, if we expanded Manhattan, just people talking about problems with addresses. And Manhattan is probably the one place where it works really well. You have avenues. You have streets. They cross each other. You can always find a place. What if we extended that to the whole world? How would it work? What would it look like? There's some code. There's some annotation of how that's done. And you can view it live. So this is, and you can scroll over and you see the exact address if we stretched out Manhattan across the whole world. There's an IPv4 map of the internet, which again uses the same mapping library, but then kind of throws in a fake projection system and stuff like that. But to map IP addresses, basically. So it's starting from, if that exists, and then going all the way up to 254 or whatever. It'll show up. It shows you kind of maps of the countries where those IP address blocks are assigned to. And I mean, this is just a fun thing. It was more a demo than anything actually practically useful. But you can search so we can search for asgeek.com, for instance. And it's going to show us where in the map of what IP addresses it sits, is it sitting next to, so it's taking a little time. So maybe I'll come back to this. Just a quick other example, just the fact that the same library has been used in other places. You'll see it look kind of similar, but this is a project with the UN, mostly in Iran kind of dealing with UN projects and mapping out the details of the different projects. And again, similar, it allows you to filter by multiple things, including on a map, and get some desalination project in Egypt. You can double click on it and get details. Everything's kind of slow. So that's just a quick demo. Again, if you guys are interested in the library a bit more, it's oxjs.org. And that has links to download all the code to get on the mailing list, to our chat room, et cetera. OK, second thing I'm going to demo. So I'm jumping from one thing to another. I'm going to see if there's a thread that connects all these three things, but maybe there isn't. So this is the company that I'm working for, for part time now. It's called Topomancy. They've also paid for my flight ticket, though. So we do a couple of things. The project that we're working on over the next year is something attempting to build a gazetteer, like an open-place name gazetteer that collates kind of open street maps, geonames, data, et cetera. That's something we're just kind of starting on. This is a tool, one of the people we worked with, Tim Waters, built a few years ago, and now we do kind of custom installs and stuff for people. I'll just show you how it works. The idea is that if you have raster maps, like if you've got an old historical map or a printout of a map, and you want to geolocate it, you want to be able to overlay it on existing maps. You want to kind of be able to play around with it in an actual geoecosystem. So you've got this image of a map that you've scanned, that you've found online, et cetera, and you want to give it real lat-long coordinates so that you can overlay it on actual maps and deal with other location data. So this is a really simple tool. You come to mapwalker.net. It gives you an overview of all maps. You can upload your own map, which is just a JPEG file. So last night, I found this historical map of Bangalore at Lawrence's house, who I'm staying with, and we scanned it in, and I uploaded it. So I'm going to need someone's help who knows Bangalore a little bit. So this map is, I think, somewhere in the 1900s, and this is the current open street maps. So the idea is you find the same point on your historical map, or whatever JPEG map you have, and the same point on an actual map. In this case, it's open street maps. Hopefully, we'll have a Bing layer added soon. It's using currently a really old version of open layers and there's all sorts of problems, but hopefully that'll be right to fight soon. But the idea being, so I've got, I'm going to probably do this really badly, but I've got an idea of how I'm going to here. I'm going to draw a point here. Wait, I need to flip the drawing tool. Let's zoom out a little bit. I think this is the Alahanka. So I drop a point here. Come on, drop. And then I drop the corresponding point here. It happens. Interesting. The problem, maybe I opened this tab like three or four hours ago because I was worried about internet and I might be logged out or something. I'm just going to, I mean, so the basic idea is really simple though, and it really takes, it's taken people like about 15 minutes of training to get started, like architecture students and things we work with is just, can someone just look at this map and be able to tell corresponding points? Okay. So I'm going to have to wait for it to zoom in and stuff like that. But the idea being, okay, point tool, double click, double click, really doesn't work today, huh? Okay, so you're going to have to imagine this works because it does work really well and I'm going to show you where it outputs. The idea being, you double click a point there, you double click a point there, you get a little marker, you say add control point, you do this for three or four points, and then you say warp image. You say warp image and then it basically, does all the heavy math, taking projections into account, et cetera, and gives you a map that's being warped on the actual map. What I am going to do is show you an actual result. So here you can see a map of Bombay that's someone warped. And then there's literally so, so once you're done warping, it exports it into a whole bunch of formats. It'll give you a WMS feed, which is really easy to overlay in any mapping client that you're using. It gives you a geotip if you just want to kind of download the whole thing or you can use a Tiles URL to plug it in. And then this literally once that was done, building this kind of overlay, this is something that we're going to get into the workshop tomorrow a little bit, using this really nice mapping library called leaflet and that UV is going to show a little bit, but using it literally it took me about 10 minutes to kind of have this really nice overlay. I can turn, oh look at the opacity slider, oh God. For some reason jquery.com was not your thing, so my jquery wasn't coming in. But you just have, this was just, it was a complete hack, this thing, but it totally trivial to then have this map, have an opacity slider, mash it up with four other maps. These things become kind of not only possible, but quite trivial to do, that if you have maps of like 1850, 1900, 1950, to just have a two box saying show me a map of this year, turn on the opacity, see what it looks like today, et cetera. It can be really nice, and this tool makes it really simple to do. And this was developed by Tim Waters. The code is on GitHub, TimWaters slash mapwalker. And you can go to mapwalker.net and see any details. This is built on Ruby on Rails and does require, it uses GeoServer and stuff on the back end. If people are not familiar with that, it's just a kind of pain to set up, but so for open, for free maps that you just wanna upload, please use mapwalker.net. And if anyone's interested in Dranzeligry or something as interested in a custom install, you can get in touch with us. Okay, third project. This is something we've been working on in Bombay and something that you can ask me complex questions about because I have been working on the code mostly, is to do bus routing in Bombay. So we, someone we work with, had access to the BST, which is the bus company in Bombay to get their raw data. So the idea was that, oh, we're gonna take that raw data, oh, yeah, they've given us spreadsheets. I mean, it's Excel spreadsheets, right? How bad can it be? It's rows and columns of data. We're gonna kind of just process it a bit, plug out GTFS and build all these cool tools. Unfortunately, it took us about three to six months just to kind of get their Excel tables into a relatively clean. All sorts of problems, I don't wanna get into it. They had names of their stops. They referenced another stop table, but the names weren't the same. So we had to do fuzzy string matching to kind of connect the stops in this table to that table. I don't know if anyone's dealt with government data before. It would be interesting to kind of talk. It really, at the end of the day, it felt like it would have been easier if we just selected the data ourselves. Just coming to, so we were outputting GTFS, which is the general transit speak feed specification, which used to be the Google transit feed specification, which it's a great format, but again, is a bit Western centric in that it expects buses to have fixed timings, which is a really weird concept here. So these are the kinds of issues we kind of really faced because even the BST and their data, they had these vague notions of timings, you know, and they would be missing for half the places and we'd be like, okay, sir, can we assume this? And you know, they were really not sure and they're like, you know, at the end of the day, we don't know what, you know, it's just they've got a vague idea that, okay, we'll release about 10 buses between 1 p.m. and 4 p.m. But they don't have fixed timings and GTFS wants you to give timings. It wants you to give a fixed time for every trip. So this is something we really had a problem with. I really wanted to kind of hack it up and just do something that's, because it's not accurate anyways. This one guy we were working with spent a lot of time trying to make sense of the BST data, which, so, you know, we had to output this while knowing it's a lie, basically, but we had to do that to get all these other tools to work. So what we do have now is a kind of simple SMS app that we've developed, where you can send a kind of place name to place name and it'll give you bus routes. It does only kind of direct bus routes now because we didn't get into kind of writing an indirect routing algorithm that's actually really complicated. We have, so we plugged in our GTFS feed into OpenTrip Planner. OpenTrip Planner is kind of free software that again reads GTFS and gives you this kind of great routing interface for free. So just to kind of quickly demo it, I can click a point in the city, so I right-click as a start trip here and it's marked it in, what's going on? Is it like a, is it my like screen resolution? I right-clicked here and it marked it. Which, let's try that again. No, so let's try it here. I don't know why that's like screen resolution or voodoo, some weird like, obviously that was never worked in presentations, but this is a bit ridiculous. Okay, great. So I say start here. I mean, obviously you can zoom the map and stuff like that but I'm just showing a quick demo. Do a plan your trip. It gives you weight changeovers. It gives you alternate routes, different changeovers. It really confidently tells you 3.12 p.m. because we had to give our data that way. Of course that's a good lie. But, so the problem we've had with, I mean it's been something that's a bit, and I'm interested, it would also be interesting to have UV kind of shed his experiences. I don't know if we're gonna have time for that running bus truth study and where they didn't work with the government agency and I think almost ended up doing a better job because of that, because they just kind of add our collected data and did something a lot simpler. The other problem with OpenTrip Planner is that it's a real black box. Like it's really cool what it does. Like this looks really cool. Like you can get directions but it's really hard to really customize it all. It's just like Java engine that if we had to change even one line of code personally, I would be really, really scared. So we've not really, and also this was mostly a volunteer kind of driven project. And so it's also the challenges of kind of dealing with that that it's kind of people are just into kind of providing some better bus routing data. I think we've managed to get something up and then the question is also can we, is it something interesting to work together with the people in other cities? We also have a bare bones Android app which you can find at somewhere. We've got a website that should be up pretty soon but it's gone through like a major design process and stuff like this. But there's an Android app that if you're working with people in Puna as well who maintain the Pune GTFS feed and seeing if we can plug in the Android app and perhaps the SMS app. Yeah, some technical details for the fuzzy stream matching. We're using a Postgres with Trigram indexes which is actually really, really good and really easy to set up and makes a really complex problem feel really simple where you have to kind of match for inexact kind of spellings. It's just something you can install into Postgres, add your indexes and do queries and it would return you similarity matches with scores, et cetera. That's really nice. I'm actually close to him so I'm gonna end here and take questions. Sorry, there was kind of a quick run through through multiple things and. You looked at Mumbai Navigator, it's a project by Ivy Hobbit. Yeah, unfortunately, I mean, unfortunately it's something that they didn't really maintain and the data was really old as well. So I mean, I think four or five years ago I met the student who kind of built that and then I think it was still up on their servers but it would sometimes be up. So again, it was something that like a student just kind of built in two or three months and it was incredibly useful. I think it was the only thing that people could actually use. So we have built a process now that when BST sends us fresh data every three months we've got a set of scripts to kind of import their fresh data and stuff like that. And most of the work kind of ended up being that because their data was, I mean, in theory Excel spreadsheets but kind of really messy. One other thing about the project was that they were able to alternate routes. So you do this like with the metro or the locals and then catch this bus. So they had the heavy math behind it figured out. So OpenTripPlanet does that. So OpenTripPlanet if you give a GTFS does the heavy math for you. So I'm not sure, I think his name was Alok Jad of the student who developed the navigator thing. I think he might have done a bunch of that heavy math himself which was pretty cool. What we do want to do and hope to do by the end, soonish is plug in the Bombay trains into GTFS as well. Now, since we have some experience output in GTFS, trains actually do run on time, trains are actually a lot easier to get. So it was kind of like, okay, we'll deal with that once we kind of get this sorted. So hopefully then we'll have trained data as well which is when it gets kind of more useful. I'd be interested in knowing what Google is doing for its Bombay or transit data actually. It's fairly decent. I think our data might be slightly better. I'm not 100% sure. But yeah, because we got it straight from BST and that was just, it took a lot of data processing to actually make it useful. We built a couple of tools around that which put this tool to map out all the bus stops because we needed that and stuff like that. So that was some tool building behind the data collection as well because we needed to geolocate bus stops and stuff like that. Anything else? You showed the whole thing, right? So is there a specific way to scan it or something? Or can you just scan it? Just regular any scanner, just get a JPEG off the map and whatever. What about resolution? So I mean, better resolution is good. I mean, then you can zoom in quite a lot and see details like here. This is quite a high-res map. I mean, it would require the internet to kind of, I mean, again, high-res is good. Low-res is not, it's gonna not be so great but it depends on your use case of it. So, put your feed to kind of upload whatever you want. I mean, you can also use it in quirky ways. I mean, hand-drawn maps or, you know, from the presentation in the morning. You see, these slightly quirky maps also can get interesting if you overlaid on actual geodata and see what happens or, yeah, I mean, this is fairly high-res. So you will see the details show up once it re-renders everything. High-res is good but anything is fine. I'd like to know how the movie world is driving a lot of tourism traffic. So you could switch to your Padma product. I can show this, you might see this in some of the movies. Sure. So the thing I'm trying to find the dots here is, is it tagged? Let me rephrase it. So I do this movie clip or a whole film but a certain sequence or let's take the song for shot in these multiple locations. So are those locations what you're trying here? Yes, yes, it's at the clip level. So you mark it in point and out point. So you save from, you know, four minutes, 50 seconds to four minutes, 55 seconds. This was the location. So it's tagged at the very specific clip level. So on 0xdb passes subtitle files. So it's the length of the subtitle. That's generally the length of the clip and on Padma it's kind of arbitrary. So people, you can mark out like if, you know, if it was a video of this conflict, talk the entire thing would be whatever Terry Bangalore but you can also mark out very specific segments. So what you're seeing here is clip level. So what you're seeing, that here is like the five seconds where he said my training is over and I'm posted in Mumbai, I'm coming today. So this is just that five seconds that was tagged with Mumbai, which is what I clicked on. Okay. So how do you get this geodata, the mxdo? Oh, just passing subtitle files. How would subtitle is being a geota? So there's a list of, there's a white list of location names. Okay. So we're not to use Mumbai because if I were to take a song sequence that's what people say, I want to go to Switzerland. So the thing is on 0xdb it's all automatic. So there it just passes the subtitle files, right? So the word Mumbai is mentioned. It knows Mumbai is a place because there's a white list of place names and then it tags this to the actual place. On Padma, it's a bit more handcrafted where people will look at that clip even though it's not mentioned, they know it is short here or it's referring this place and people will manually add the annotation of that place. Well, there's some standard bits kind of geotags as you keep shooting. Who do you know? There was some mxd or mxd? Ah, I see, like parsing it from the camera. You know, good question. I'm not sure. When grabbing video now, there might be something. It's a standard bit. Yeah, but the thing is most of those cameras are the next time, sir. But that's, yeah, on still camera, on video cameras, the cameras that we've been dealing with and especially, I mean, we used to deal with tapes. Now there's more kind of like HDB digital that was never kind of geodata on the tapes themselves. But no, that's a good question. New cameras with GPS, this would be a good thing. Yeah.