 Okay everybody, sorry about that brief delay. Welcome and thanks for attending one of the late sessions of the last day here. We're very excited to have you all here and we're very excited to talk about the exciting things going on in Drupal mapping and hopefully get you all excited as well. I'm Nate Parsons and I'm gonna be sort of emceeing this event and I am the least lit of these luminaries. I have to say these are the exciting guys who are sort of moving forward with a lot of the Drupal mapping and they can share a lot of really interesting things with us today. We're gonna start off today talking with Rick about what the state of Drupal mapping looks like just from a high overview and then we're gonna get into some of the exciting things that are being done with Drupal mapping today. So Rick, can you hand it over to you? Thanks Nate. Hello and welcome to my presentation. My name is Rick DeBoer, I'm from Melbourne, Australia where I run the Flink Collective where we continue to have great fun with Drupal and Maps in particular. You've come here because you want to take your next step into Maps. Maybe your next step is to create your first map or maybe you've been there, done that and you are ready to cluster 100,000 markers. We've got all these bases covered with this wonderful team here. For my many presentation, I hope that you take away two points. One, a handy map starter kit and two, a desire to take your next step into Maps. So let's begin. Just like you can't have a school bus that goes at the speed of a Ferrari or a Ferrari that seats 65 people, you can't have everything in the one map. So you have to sit down with your client, listen to their requirements and prioritize the feature list and balance that against the time frames and the costs. And once you've done that, you take your feature set to Drupal.org and you look at the project pages and the documentation pages and you probably go like, ah, so many modules and they all have similar names as well. It can be very confusing. You see, Drupal map land is a bit of a jungle. I try to make a diagram here, but it's not nearly complete. What it shows is it's a mess. I try to bring a little bit of order in it with on the left-hand side, we've got a silo for the coordinate storage modules. And on the right-hand side, we've got our map rendering modules, the big ones being Google OpenLayers and Leaflet. And in the middle, there's this bit, usually views-based, that takes your locations from the left, turns them into markers, applies the tooltips, the color coding and all the rest of it and superimposes them on your map tiles. And so your job is to find a path from the left through the middle to the right. Problem is, many of those paths don't really, don't exist. And the situation is further complicated through these, what shall I call them, auxiliary modules that do handy stuff on the input side like geocoder or beautiful stuff on the output side, like marker clustering, and they all commit their own constraints and so that you can't have any this with any that. There is, however, one module. Oh, to give you an example, say you've got the location module and you're using a GMAP and it works fine, nothing wrong with that setup, but then you're ready to put some fresh new lipstick on that old Google face. And you've read about the leaflet marker cluster module and you think, this is cool, I wanna use that. Leaflet and leaflet marker cluster only work with geofield. There is a module though, oh, a slide has gone missing. That's okay, there is a module which is right in the center here, I can't highlight it, it's called IP Geolocation Views and Maps that allows you to use any of the four input coordinate storage methods, modules, to connect those to any of the three map rendering engines. And so it's no surprise that I recommend that as part of the starter pack. But don't just believe me, on my journey through map land, I've talked to a lot of people and they've come up with similar solutions and I've asked them whether they wanted to be on the slide, so I'm sure they'll be more than willing to share their experience with the Maps data kit or a very similar setup. So in closing, for my Maps data kit, I would recommend geofield for your latitude and longitude coordinate storage, views because you can't avoid it, leaflet because it's lightweight and optimized for mobile, and IP Geolocation Views and Maps because of the reason I just mentioned, but also that it brings a few handy features of its own, including some of the context sensitive stuff that Rhys mentioned where he said if I go to a website it should know that I'm in Denver and then do something with it like put up an ad or put the visitor marker on the map or stuff like that. So what's not to like for your next step into Drupal Mapping Lab? Thank you. Hi everyone. I'm Paul Delayera, Drupal on Twitter, and I'm gonna show you, I'm gonna talk about open layers. I'm here to show you that open layer is not dead. It's still maintained for one years now. I'm the maintainer of open layers. And I will show you the new stuff and the upcoming stuff in open layers, 2.x and 3.x. So first of all, the 2.x branch is the stable branch of open layers, the one that you should use if you start mapping from today. The 3.x is still highly experimental and it's still under development. So as we only have five minutes per person here, I cannot make live demos so I will only show you screenshots but if you want to talk to me after the presentation, I will show you those demo live on my computer. So the first new stuff in open layers is the Unify Admin User Interface. So if you install open layers for the first time, you'll see those tabs, layers, maps, projection and styles and you'll see that they all have the same aspect, the same style. So this is the layer page, the maps page, the projection page, the style page. So we are using Ctools Export UI and this allows us to create those administration page really easily and it's really clean in the code also. So second thing is the behaviors of, the behaviors display improved into the map edit form. So it is the old way of displaying those behaviors in map, in the map edit form. This is the new way. It's cleaner, it's better and there is really less code than before. There is a major cleanup, new pop-up style. So this is the old way of displaying pop-ups into open layers which is a white pop-up with a red button on the right, top right. This is the new way. This is a bit more beautiful, this is modern. Next, the new behaviors. This is interesting. So you have first the graphical behaviors which allows you to display parallels and meridians on a map. You can customize the color of the lines, everything. You can customize a lot of stuff and you can display it on any map you want. Then the hover behavior. This is when you are displaying features on a map like this map, you have a couple of features on it. When you pass your mouse on one of the features, the hover behaviors allows you to change the icon and replace it by another icon. This is really handy. The compass behavior, see on the top right of the map, you have a compass and when you pan on the map, the compass changes the orientation. You can change the icon, you can change a lot of stuff also. The refresh layer strategy behavior. This is a map provided by another module for open layers which is called open weather map. It's displaying information about the weather and so on. And this map has the refresh layer strategy behavior enabled, it means that every 16 seconds, the layer will refresh automatically and you will have a beautiful animated map of the weather on your site. So the save and edit button, this is a small feature but this is a time saver when you are working with maps. So you are not going back to the map list when you hit on save, you can click on save and edit and edit the map again. Dynamic markers. So dynamic markers is when you are displaying markers on your map and you want the image to be customized. So for example, this map is displaying two markers and the image of the markers, the red images, are coming from a field from an entity. And you can change the size of it through the image styles of Drupal and so on. I told you it was faster. So the new layer type in open layers. Bing layer type is now into open layers. It has replaced the virtual hearse from Microsoft. Mapbox layer types is also into open layers. You don't need any more modules to get them working. And an image layer type, it means that you can upload an image as a layer into open layers. Another new stuff is the contextual links. So when you are editing a map, when you hover your mouse on the map, you'll see on the right some edit links. So you don't have to go back to the administration interface to edit the map. You have all the links on the right, edit, clone, export and revert if you have edited the map. Custom projection. This is a big new features that has been implemented in open layers by a friend Augustus Skling from Germany. And it allows you to display custom projections on map. Maps with custom projections. This is it. And of course, Ajax. You can now display your maps using Ajax on forums and on pages. And unfortunately, there's no demonstration about that. If you want, you can talk to me after the presentation, I'll show you all of that. Well, that's a bit of everything, the new stuff in open layers. Yeah. This is a beautiful day. Hi guys. My name's Tom Nightingale. I am from Vancouver, BC and I work for Affinity Bridge. So I'm gonna talk about a couple of techniques we commonly use across all the libraries for visualizing geo-data, the formats that we use. And I'd like to talk about some advanced interactivity techniques that we've been working on at Affinity Bridge. So typically when we are visualizing, we've got a bunch of geo-data that we've collected somehow put into Drupal. There's two types, two methods in which we can send the data to the browser and render it on screen. The most common approaches would be vector data rendered as an SVG graphic. Or you can also rasterize your data into a bitmap image and serve that up as tiles. Vector data can come in a variety of formats. There's more formats than you can count. But more often than not, when we're dealing with the web, we tend to stick to the plain text formats such as WKT, well-known text, Geo, JSON, Keyhole Market Language, KML, which both our main mapping libraries, OpenLayers and Leaflet, have or provide support for. Vector data's great because it's like the raw data. It's very easy to process, manipulate, style. But there are some downsides too. It can be very, it's the raw data, so it can be very robust. I have a very large footprint in sending over the wire to your client. The rendering is done client-side, which can be a plus for your server people and a boon for your end users. And so typically when we have very large amounts of data, the best way to get that into our client's browsers is actually by rasterizing it into bitmap images and serving them to the client as tiles. Once that data's been rasterized into an image, web browsers handle images very well. The rasterized image is a consistent size. It's just your typical 256 by 256, so it doesn't matter how much data you've stuffed in there. You also benefit from images being fast to load, cacheable at the HTTP layer, that kind of thing. So moving on to interactivity. Traditionally with our Drupal modules and with maps in general, interactivity's kind of been limited to putting markers on a map. We've got a bunch of point data. We've rented them as markers on a map and we have little pop-ups when you click or hop on your markers. I mean, that's effective, but it's a little overused, I think. And with the state of JavaScript today, and the other JavaScript libraries on the internet, there's a bunch of other really cool stuff we can do. So one of my favorite libraries for data visualizations is D3JS. D3 stands for Data-Driven Documents. It's a library developed by Mike Bostock, who does some really cool stuff. And it's a really useful tool when it comes to creating really interactive or some SVG-based graphics. I've got a couple examples I'd like to quickly demo if they will work for me. Not that one. This one hasn't loaded, let's try it again. Here we go, so this is really big. This, we're just gonna look at it. I'll zoom out a little bit. I've loaded the, that's not what I wanted to do. So in 2010, in September 2010, in New Zealand, in Christchurch, there was a very large earthquake. I think it was 7.1. Magnitude, and it caused a lot of destruction to the city of Christchurch. They're currently rebuilding at the moment. The New Zealand government has released a bunch of the information collected about the earthquakes and that you can download and consume. And I've sort of pulled all of the earthquake data of earthquakes that are greater than 3.0 magnitude for the month of September. There were a lot of aftershocks after the big one. And you can see I've rendered them on this time series along the bottom of the map here using D3JS. The time series is interactive. We can click and drag. And as we drag and create sort of a date range, we can sort of drag through and filter, filter and select sort of query the data I guess, filter by range. Now this is all outside of Drupal. It's just what we're doing in JavaScript. And then we can also click on individual earthquakes and sort of see some information about them. So that's D3JS playing very nicely with leaflet. I've also got another data set from the city of Vancouver and someone's gone out and collected information about all of the graffiti in the city and where it sort of most prominently occurs. And I've fed that data into D3JS. And you can see here, I've used a technique called hex binning to dump that data into buckets based on proximity. And you can see where the hexagons are darker red. That's where there's more frequent occurrences of graffiti in Vancouver. It's a good way of, it's kind of like a heat map of visualizing your data. Thanks for the presentation. Finally, at Ifany Ridge, we've been working with a non-profit organization called Peace Geek, Peace Geeks. They have a mission to strive to empower organizations promoting peace and accountability and human rights with technology tools and training. They've been developing a distribution for Drupal that collects and maps instance for various locations and we were tasked with building a sort of a mapping component that had some features. They wanted to see some clustering. They wanted to be able to filter on the data. The data had multiple dimensions to it. They wanted to be able to configure it as well. Their data model wasn't set in stone. So I've used D3JS and another library called Cross Filter which provides very quick and efficient sort of slicing and dicing of the data set to produce this incident map we've got here. I believe it's in reports. What should it mean? Well, it's quite prepared. Should load. And if we zoom the bulk of this, there's just sample data in here at the moment. Most of it's in Uganda. If we zoom in here, we can turn off the clustering and you can see we've got a set of data points. And similar to the Christ Church map, we've got a timeline filter on the bottom here which I can click and drag on. Whoops, works much in the same way. And then on the right-hand side, whoops, can I scroll sideways? Oh, it's not gonna let me. That's disappointing. You can see, can I do it? Oh yeah, right, I see, if I just click up there. There we go. Down the sidebar here, we can also filter by other dimensions on the data. So this is actually just a taxonomy vocabulary that we're filtering on terms that the data is tagged by. As you can see, as we filter, the map is updated and the totals down the right-hand side here and the timeline at the bottom are also all updated dynamically. And that's the kind of stuff we can achieve with JavaScript because it's super awesome. And that's all from me. Thank you very much. Hi, so my name is Patrick Hayes. I'm with Stanford University and I'm gonna be co-presenting with Joseph here. And we're gonna be talking about kind of more advanced mapping techniques. We've both been, along with David Smiley, who works on Apache Solar, been developing a technique for building maps that scale to millions or actually many millions of data points and actually be able to represent that data to users. So basically the problem is how do you build a map with millions of data points? And traditionally what Tom talked about was rasterizing it. Generally what you do when you wanna build big maps is rasterize in a minute tiles. And that is great, but it doesn't work if you're constantly updating that data, if it's real-time data, or if you want it to be searchable or filterable by end users. So there's basically two problems here that we have been working on a solution for. And the first problem was cartographic. How do you graphically show a user with millions of data points? And this is not a good answer. So we're gonna be showing you two different ways of making a map with millions of data points or hundreds of thousands of data points that actually shows all the data and yet also makes a really clean interface. The second problem is technical. How do you actually scale to millions of data points while keeping the map searchable and filterable? And basically trying to make it live and in real-time. So the answer to these questions is to use a search engine like Apache Solar. You could, the same methodology could be applied to using Elasticsearch or Sphinx or any other search engine that has grouping or clustering abilities. I think he'll show you it with MySQL too. It's a little bit slower, but the fundamental idea works. So the basic idea is to use a Geohash grid. And so what Geohash is, is basically, it's two different things at once. It's a grid of the whole world. We slice the world into 24 bits and we take each of those slices and we subdivide those into more. So basically down here is an example, Geohash at the bottom, U4PRU. And what that means is that each character along that Geohash is making the grids smaller and smaller and smaller and smaller and further and further and further subdividing it. And once you get to about a length of nine Geohashes, a length nine Geohashes about the size of a pizza. So, which means that you can actually use Geohashes as both coordinates, really long Geohashes, but you can also use Geohashes as bins. So at shorter things, you could think of Geohashes as bins that you can fit stuff into. And that's exactly the property that we take advantage of in order to index or stuff into either solar or my SQL. So basically what you do for any given record, you create nine fields, one through nine, and you simply take the longest Geohash and put that into number nine and that works as a coordinate. And then you go all the way up, getting less and less and less accurate, but most interesting, more and more and more points fit into these common bins. And then once you have that, you can use faceting. So, for example, depending on your zoom level, you're gonna want to facet on a different accuracy with more or less bins. And so basically what you can do is you, when you're displaying the map, you query solar saying, give me the top 50 results, but also facet on these Geohash bins so you get a summary count of everything that the user is looking at. And I'll show you what it looks like. And then finally, once you have your facets into these bins, you can retranslate those back into Latitude and Lunge2. And from there, I can show you a live demo of what this looks like. So, let me zoom out. So what we have here is the green is a actual results coming from Apache Solar and the red is a heat map of, this is a database, this is a data set of about 50,000 results. And so we show the user the top 50 most relevant results, but we also show them a representation of that other 50,000 that they're not seeing as this heat map. And so if I do a search for, for example, Earthquake, this is a database of peer-reviewed articles written about geological topics. See, this one will work. Internet's a bit slow. There we go. So for Earthquakes, we can see there's a lot of results, but interestingly, if I go over to, so the green is the most relevant results, which are fully interactable, and the red is the summary data which we get from these facets. And so for example, if we look at Taiwan over here, we see that it basically, the red invites the user to zoom in to see more data saying, we couldn't present all the data to you because there's just too much data, but what we can do is we can suggest that there is more data here to find. So then if I do zoom into Taiwan, we actually get all the green data points which presents the actual data that we're interested in. So, and Joseph is gonna show you another example of this, and it's just the same methodology, but just a slightly different visualization of a way of accurately displaying millions or hundreds of thousands or millions of data points. Yeah, so hi, I'm Joseph Davonic, that's you. And I've been working basically on the same topic the last year. So let me get the presentation started. Yeah, so this is just another way of representing, in this case, 100,000 items that we put on a Drupal jobs recruiter installation using server side clustering. And this is how it basically looks like. So whenever you zoom out, points will be aggregated on the server side and the JavaScript library will just issue a bounding box request in order to fetch the data that is available on the current viewport. And the server side will, based on the configuration, return the results in order to allow the user to drill down the data. And now we are in Portland. And we are seeing, in this case, I'm just outputting the idea of the items, but definitely, as you've seen before, you would just do a pop-up and show all the data that's relevant to the user. So how did we go there? I was writing my master thesis on server side clustering with Drupal. And so I was researching cluster analysis, geo-visualization, and came up with the ideas that already were talked about in the solar community. So, as Patrick, I decided to use geo-hash in order to group items based on the zoom level and take advantage of the gradual precision property that geo-hash has. Of course, Drupal is a great contrib space, and we have already seen that there exist quite a number of modules. So the second task that I would take upon was integrate into the existing Drupal stack. So this is basically what they came up with, I will not go into details because we're very short on time, but it basically integrates in geo-field, views, geo-json, views, search API, and solar. So that was quite a big task, and this is positive at some point, but it also requires a lot of glue code to get it working. The performance evaluation shows that a damp PHP implementation of the clustering algorithm clearly won't work. The MySQL clustering already works up to 100,000 items, then the clustering within the database will get slow as it's per request. We don't really store the clusters themselves, we just have the hierarchical spatial information present. But fortunately, we have great tools outside of Drupal like solar, and so we can have quite performance clustering with the solar engine, and there's already cool connectors like search API and Apache Solar for integrating Drupal into the solar engine. Finally, I would like to share some lessons that I've learned during this journey, writing Matisse's on Geocluster, which is integration is great because you can take advantage of so many modules. You can use leaflet as a visualization component, but you could also use open layers or G-Map, that's perfectly possible. But at the other hand side, Drupal really shines at being a content management framework and if you process real big data in Drupal, things are getting slower and slower, especially if you try to integrate with views. You guys might know that loading fields and loading entities is really slow, so keep that in mind when working with data in Drupal. And as Rick was saying in the very beginning, you should really think about the story you wanna tell and how the use case will work out and not just have a look at what technologies are there, but rather get your story clear in order to make customers happy and make a successful business, of course. Yeah, so prioritize. You can also rasterize data, have a look at Mapbox and that's for me, thank you. Hi, so my name is Brandon and I'm a developer at Phase Two and I'm one of the maintainers of Geofield and a couple of other mapping modules. This is my co-worker, Nate. He is also obviously as a co-worker would be another developer or not developer. What do you do at Phase Two? I do a lot of coding and I do a lot of sort of figuring out what our clients want out of things and so Brandon and I work really closely, especially in the mapping space to sort of figure out how to tie these things together and one of the things that we're sort of learning is we kind of get more and more involved in doing interesting mapping projects for our clients is that there are not that many people in the Drupal world who are helping to work on mapping but they're doing some really interesting things and so part of what I wanted to do is get involved to sort of help bring some more sunlight to this and sort of show all the cool things that these guys are doing and also get some more other people to help join the cool kid community here, so. Yeah, definitely, I mean we really don't, I mean I can't speak for everyone I suppose but we don't really want to be the cool kid community club. Like that's kind of dumb and we want to have other people come in and join because I mean frankly maintaining this long list of modules is kind of hard and we more hands make lighter work basically so what we're kind of looking for is are we kind of want to talk a little bit about just community in general and how to get involved, who to talk to and then some things that we kind of want to do to expand out. So besides the people on the panel, oh and if your name's on the slide, surprise. So we have a few other people that, so is love in the audience somewhere? No, yes? I guess not, okay, so ThinkShout is a company here in Portland and love specifically maintains the leaflet module which is another data visual or mapping visualization map or module and it's a really interesting library. So Rainier, I know I've seen Rainier around, I don't. Hey, what's up, I see you. Rainier has also been involved with the mapping community for a while. He's a maintainer of the WMS module and has done a lot of really interesting work doing some neat mapping community things. And then Jeff, who's over there? Hey, Jeff. Jeff is another developer who's the maintainer of Views GeoJSON and my understanding is looking for a co-maintainer, so. Yeah, so we're all, all of us have a lot of things that we're involved with and would love to talk to you about any of the particular aspects of mapping that we're involved with. So if you're interested in making it look pretty with a visualization or showing something interesting in that, we're happy to talk about that. We're also happy to talk about, how do I get all of this information into Drupal and to do it in a sane manner that doesn't break the laws of space and time and that sort of thing. From a community building standpoint, we had a few ideas as far as what we wanted to bridge out and do. Yeah, and I think when we're looking into it, there's a couple of pieces. We want to make mapping more accessible for people because mapping is really cool and really useful and a lot of people have geographic problems they're trying to show. And this panel is from all over the world. It just sort of shows what a big problem this is. And one of the things we want to do is sort of make that onboarding a lot easier. And as Rick showed, it's a little overwhelming when you look at all the modules tied together and how they're wired and all of that. So we're really interested in sort of developing better sort of on ramps and starter kits. And that's something that anybody of any sort of experience level could really help us with. And if you want us to kind of help you walk you through your first project, that could be a great way for us to document some of those starter kits and some of the things that we as people have been doing a lot of mapping may overlook is some of the impediments. Yeah, I mean if you've ever built something with any of our modules and you had like five WTFs in the middle of something, please let us know because we may not, it may not just be obvious. Yeah, and then the other thing is the way the documentation is currently sort of describes doesn't really show you what's sort of active, what's widely used and sort of which pieces should fit together how. And so one of the things that we've been working on is sort of developing a sort of better evaluation matrix just to sort of let people know what's active, what ideas are sort of being used by many people and there might be a lot of documentation on and what things might need you to sort of step in a little bit more and say hey this is a pretty interesting and unique problem that I have and I might need to get more help around building that out as a module or helping to get the community more involved and helping maintain that or make that interesting. Definitely, yeah so I guess the very short version of it is it takes a village and we need your help and we would love to really get people involved. You don't have to be a coder, if you own a business you probably have people who are coding for you, you can have them do it. That's the easy way honestly but if you're not like that then if you're just interested in the subject in the problem space, you can go through and read documentation, you can test things, you can come up with new ideas. I mean we enjoy coming up with stuff but we can't think of everything. If there's a problem that you think might be more broadly used than your specific use case then we'd love to tackle those kinds of problems. Yeah and I think the other interesting thing about mapping is it's sort of a connection to a wider world. I mean as you just saw from all the technologies that are being used and utilized, there are all sorts of interesting things you'll get to know about. Postgres is often used as a database back end. There's an enormous variety of very powerful JavaScript libraries that come into play here and there are all these high end professional systems like ArcGIS and things like that behind the scenes that are creating the sort of source layers and base maps that are being used and imported to decorate and put your data onto and then the user interface tools in Drupal itself to actually decorate and add data to those maps and all those things are interesting and we could use your particular area of interest or expertise and you could see the world this way so we'd love to have your help and we hope you're as excited about this as we are. Definitely. Is that our last slide? Yes, so how can you help actually? I mean it's fun to say that we need your help and then not give you any follow up information but so there is a group, groups.drupal.org slash location and mapping that's kind of been the central location or it's been a location anyway for a lot of the overall community discussion around just mapping and that kind of thing and this is gonna be the touching point where we're going to start looking at revamping what's there and trying to as a community expand and just make the mapping in Drupal world a better place. Yeah, yeah, so thank you all and come grab us afterwards and we'd love to talk with you all. Thanks. Thank you. I really appreciate it. Thank you. Yeah, sure. So we do have a little bit of time and we wanted to do Q&A so if you wanna queue up here and ask questions, otherwise we can definitely talk about whatever you want. Yeah, great. I learned some things and I hang out with you a lot the last few days. I know about a year ago we had office hours. Yes. I spent some time on IEC during that. I think it would be a really good idea to here now plan a date and time for our next office hours and then allow people who are now behind me who are all gonna go to Drupal.org and download modules and experiment with it and then in two weeks from now or so we meet an IRC and we start assigning tasks because yes, there's a lot of stuff to do but we have to put names to who's doing the documentation, who's gonna do a sample install or sample distribution or sample website, whatever. We have to put names to tasks because yeah, we do have a massive amount of modules and we can do really, really, really cool stuff with it and we just have to market ourselves a little bit better but it's gonna require some work and I think there's plenty of people who wanna spend time and do work but we have to be a bit more specific after this. I know, do you have your agenda with you? I couldn't agree more with everything that you just said so that sounds fantastic and if you guys are okay with like two weeks then. There we go. On IRC on freenote.net, pounddrupal-go is I think there's a lot of discussions that happen there so if you're interested in mapping that's generally the place on IRC to come hang out. I really like the Geeks for Peace map. I work actually against hunger and that's kind of similar to that interactivity that we want. Could you give us an idea of, how I kind of manage developers, I don't do it myself. How many hours was that to kind of develop and implement? So, how many hours? In building that map, I put quite a lot of hours into developing, it's not built, so there's a caveat here, it's not built on top of the existing leaflet module. The existing leaflet module is great at what it does but it's a little bit limited if you want to provide that kind of integration with other libraries and stuff. So I have, and I've actually arranged to meet up with the leaflet maintainers hopefully after this discussion and have a chat with them. I've put quite a lot of work into kind of prototyping a maybe a leaflet 2.0 or a way that we could start doing that to basically provide, I've called it leaflet framework but provide an underlying framework so you can build some more richer, more interactive stuff on top of the leaflet module. That, all of that work is actually up on GitHub right now. It's under the Affinity Bridge GitHub account and it's called leaflet framework. So go check that out, that will certainly shave a bunch of hours off and definitely ping me if you have any questions. Documentation's probably minimal as a prototype. But as far as like to take a leaflet framework and build what I demoed today, I'd say 20 hours, 30 hours. Yeah, because for us obviously our budget are limited. So what we ended up doing is just going for an embed and using Mapbox and using a very simple, just for like an interim solution. But from my point of view, having something kind of a bit more developed like you're describing that we can plug in and very user friendly is just would be amazing for nonprofits like around the world. Yeah, totally. And I know the Peace Geeks guys are, I mean there is overlap, there's other projects out there. Usahidi is one of them and there was another guy that we talked to at the boff. I didn't get to talk to him, but I hope to meet up with him later. Who is developing a similar kind of distribution I think in Drupal. There's a lot of overlap there and I know the Peace Geeks guys are quite interested in talking with others and collaborating as well. Right, thanks. Cool. Oh, there you go. Yeah, so just like my point of view on this discussion about sustainability of mapping modules, it obviously each of us, we are developers, so we are scratching our own itches and we already do quite a decent job in integrating against each other. Like formerly if you, you really should check out the book by Alan Pazzolo and forgot the other name. Like Drupal mapping, it's already two years old, but it's still very accurate and it talks about how we shifted from very monolithic mapping systems into like an ecosystem that is basically geo-field and integration of modules that work quite well together. But what I think that what we really would need is like a process management. So somebody who's not a developer and who is passionate for mapping, come talk to us or lead that process in order to just get us going and plan the future together because yeah, we are tight on scale as always and mapping is usually just like an extra nice to have. So the project, like the budgets are not that big in mapping with Drupal. Yeah, just a slight addition to that. It's nice having the only mic over here and not having to pass that around. You know, as much as it pains me to say, you know, having eight or nine developers together, we could probably use a project manager or two. So. And I just have to say though, one of the things I'm really optimistic about, about Drupal mapping is the state of it right now is it's now an ecosystem as opposed to the monolithic location GMAP module. It seems really seems like we're in actually a really awesome state right now where we have a lot of the bases covered and there is an ecosystem of developers trying new things and also an ecosystem of developers that are committed to interoperability, which is really awesome and a little bit different than what we had in Drupal 6. So yeah. I appreciate your comment that it's very difficult to find specific answers to specific use case questions about mapping. And it seems like whenever I come to a Drupal presentation on mapping, it's always about taking points and putting on a map points of data, but in the case of a traditional like an election map, red states, blue states, I know how to do it in JavaScript, but is there a Drupal, is there a recommended Drupal way to get to that or a set of modules that we should be looking at to do something like that? Sure. So the geofield module is, it's one of the ones that I maintain and it is a, it stores WKT values. So you can drop in anything from a point to the entire United States of America. I wouldn't recommend dropping the entire United States of America into one field, but you certainly could do that if you wanted. And then from like a display standpoint, you should be able to show that either through open layer, the open layers module, the leaflet module, and if you need to do something custom, which if you're talking about like an election map at this point, you would probably need something custom on top of that. What was the module you mentioned? Geofield? G-E-O-F-I-L-V? Geofield with open layers. And that will like you take a state and then map the outline of that and have it clickable or turn change the color of it or something like that. Yes. Do we have anything that will take like census data or anything like that where you don't have to directly put it into the? Yeah, there are some, yeah. So what you could do is you could, there's a module out there called Geocoder and it can take, you can upload files like if you have a KML file, you could use that to basically you have a file field where you upload your KML and then you have Geocoder which translate that KML into a Geofield. And the Affinity Bridge folks who are sitting just here, we're working on shape file integration. So doing something similar where you can go from a shape file to a Geofield. So and I think we have support for GPX files, for KML files, they're working on shape files. So there's a lot of options up there for getting data in and we support points, lines, polygons. And I think very soon when GeoPHP goes to 2.0 we'll have 3D stuff in Geofield too. So that'll be very cool. Okay, thank you. Can I just quickly add, so that was covering a lot of the storage of the data. Traditionally like rendering things other than markers on a map has been difficult because the browsers haven't been particularly efficient at rendering lots of points. And when you get polygons, it's like one polygon could be equivalent to 100 markers and that's just one polygon. If you have every state, it can be very heavy. Browsers are getting much better though these days and you can obviously simplify a lot of these polygons if you can apply some constraints to your data. That's totally, things like the election map or if you want to do like a Coroplet map or more thematic mapping. I don't have any concrete solutions for you right now in Drupal but that's kind of where I was hoping, I'm hoping to sort of head towards, I've got some examples outside of Drupal using things like D3JS which are great tools for thematic mapping. So yeah, keep an eye on that one. And if you didn't want to, not to keep adding to this answer but you can also avoid just storing it in Drupal in general, I mean you can load like a KML file or something like that and not run it through the database in general and just plop it straight onto whatever map and that may be more performant for you depending on your use case. This is a follow up to that question and that is, is there any tool, Drupal or not that lets you just click and create the boundaries of a polygon that you could use in a Drupal solution, I guess. Tom, do you want to talk about your awesome leaflet input because that's pretty awesome. Yeah, so there have been a bunch of modules that provide basically input into a map, clicking like WYSIWYG style input but with a map. I am responsible for leaflet widget which I apologize to everyone right here. I published about six months ago and then subsequently haven't touched. I had some constructive feedback earlier today though that apparently if you bring in the, so leaflet widget uses leaflet library to handle the drawing and the map rendering and leaflet and the leaflet draw extension have been in active development over the last 12 months and so a lot has changed between when I was working on leaflet widget and where leaflet draw is at today but I heard today that if you bring in the latest version of leaflet draw and drop it in that leaflet widget module, it still sort of works and I totally plan to bring everything up to speed now that there are actual final releases for leaflet draw. So all a bit of a work in progress. I think there's also an open layers. Yeah, in open layers 2.x there is the open layers, open layers, no, geo field open layers module or is it open layers geo field? I forget which way it goes but basically it just provides a open layers powered widget for geo field that lets you click and draw polygons or lines or whatever and it's configurable. So that works today. That works today. It depends on which library. Thank you. Hey, I was really interested in the work you were doing with the binning of aggregating and clustering and sort of following up on the question about election stuff and states. What I'd like to really be able to do is aggregate based on arbitrary boundaries like a city, a county. So I would have like a shapefile or a well-known text that would define cities, counties, zip codes and be able to take point data and aggregate them into those buckets. Is there a path to that now or soon? What's the best way to attack that? So the way to do that is there's a module that I started on a fitting bridge that's taken up called sync post-gis and it's a little bit complicated. It does require a little bit of technical skills but I think it basically works. Basically the idea is that you import your data. So post-gis is a database where you can do spatial queries where you can do spatial queries like tell me I have this database of states and tell me which point this state is in or I have a county and tell me which counties this state is in and basically what you can do with that is you would have an extra field on your node saying like which state or which county or whatever and then basically when you save the node it goes over into post-gis via the sync post-gis module. Sync post-gis figures out what bin it belongs into and then sends that back and I think there are some other, is there a tutorial for sync post-gis that tells you how to do that? No, documentation is really lacking. So apologies again. So maybe this is a good indication that we need to better documentation around sync post-gis. It's mainly kind of there for developers so you definitely don't get a point and click solution but kind of it is a problem that has been figured out and with a bit of ugly grease you should be able to get it done. Great, thank you. So Geocoder seems really promising. I haven't used it before. But are there, is it a good solution for importing and being able to get geolocation data from like address fields or something like that? Yes. Is it, does it have any limitations? Like is it a PHP library that kind of connects to the API or something like that to some external resources? So the service it connects into, I think it's pluggable so it just uses a C tools plugin to say I think there's like seven different services now, something like that. So you can go to Google, you can go to Yahoo is gone now. There's Bing. So there's Bing, there's Yahoo, there's Yandex if you're in Russia, there's, MapQuest, yes, there's a MapQuest and there's a few other specialty ones for like pulling, like except to add off of photos and that sort of thing. Yeah, so. From my research on, let's just say Google, like I know they have a restriction per 24 hours, it's like 2,300. But a lot of these imports that I'm hoping to do are gonna be bigger than that. Are there ways that we can kind of get around that? I mean, you guys have hundreds and thousands, like how did you get- Don't use Google. Yeah, or pay. I would love to hear, do you have any suggestions for paid services or anything like that that even if it wasn't paid libraries, anything that we can use to help get that location data? I think there's like one approach that you shift to client side geocoding basically. Sure. So the clients will use their own IPs and that will get you off the limits. But if you really need to import big, big data and geocode it, I think you'll just go for a professional service, right? Yeah, at phase two, we did a project a while ago where we had to deal with a similar problem as far as like importing a lot of data and dealing with the geocoding. And what we basically did was we batch processed it so we would only do X number of imports per day. We had the luxury of being able to do that over the span of like a month versus having to do it every day. It's not the best solution in the world but it got the job done for what we needed. I would echo via the client side if that's an option for you. It doesn't sound like that's your particular use case. Am I reading that wrong or? Honestly, I'm really hoping to use it more with like retail locations for brands and a lot of these brands have over 12,000 locations. They usually want to be able to do the import and export themselves. So like I can map it in, the geolocation is where I die every time. And I'm like, let me help you out. I'll dissect your feed, chop it up and then I try to run it and then I'm kind of in this process. It sounds like you have an actually a relatively similar situation that we were talking about with our client where like, I mean, like do those locations change all that often? Once I'm in, I'm good and I've got a great interface for being able to update really easy with Ajax and things like that. But now it's mainly just the import export that you usually do it only about once every quarter, if that. So it's really like that. Those use cases once every month, but each one of them does this. It's like routine death for me. One of my pet projects that I kind of want to do in Geocoder at some point is implement caching to help with that. So if you have, you know, like, if you're trying to encode like the same place several times, then that would help with that particular thing because you wouldn't ping Google every time you would ping your own cache. But my advice in your situation would be to do the batch processing. That's not something that you would have to do a little bit of elbow grease on your site to get that code going that hasn't been pushed back out and apologize for that. But yeah, I would try to batch process that all over the span of a few days or weeks or however it takes to get that done. Okay, and besides just that version, do any of you have any experience to recommend any paid service? Besides the $10,000 a month G Google geolocation service? I think, do we support G names, or not G names? Ask that man back there. And they don't have really limitations on it? So basically be on the lookout for it. You guys are kind of actively working towards a module for it, is that what I'm hearing? That would be awesome, I'll see you. Hey, thanks a lot you guys. Yeah, thank you. Well, I just wanted to say thanks. The scene what you guys are all doing in different ways with mapping is really incredible. With the Apache Solar example, that's something that we're playing with for the first time, I just didn't know if there was a good resource for Solar Meets mapping for the demo, like I saw the URL link was crazy long. I didn't get it down, I just didn't know if it's on groups, if it's on a Drupal group but just trying to... Like my thesis contains hundreds of resources linked and in this particular presentation, I've linked to the presentation by David Smiley who is a Lucine Solar developer. He's talking about Solaris. Am I getting the, what is your particular interest? Mapping a lot of data, not caching using the mapping Solar integration. My solar experience is not super extensive. Yeah, so I think David Smiley and his presentations or get a hold on him and the two of us. That'd be great. Outside of that too, I'll make sure that slides and all the links and stuff get somewhere online. We'll probably post them on the DrupalCon site and then on the groups.drupal.org. I'd also like to thank all of you guys for stepping up and especially when the DFC guys moved on, I really appreciate all the work you've been doing in there. Can you tell us a bit about Mapbox Tarmil and whether it has any relationship to Drupal, any way that it could be used? Does it build static, is it a tool for building static maps? Yeah, oh, sorry, how about it? Jump the gun. So the Mapbox approach, I touched on the raster approach, you know, rasterize your data into tiles and serve that up. That is the angle that the DFC Mapbox guys are really going. I think where they really parted from Drupal and where their solutions aren't so ideal for Drupal is Drupal's a great content repository and we've got great crud, right? We can create, update, delete over and over again which actually poses quite a challenge for mapping stuff because there's quite a lot of processing over here involved with putting these things on a map or doing things like intersection calculations or whatever. So to sort of pull all that together, the Mapbox approach doesn't work so well with dynamic data. But could you use, let's see how to specific use case, let's say I don't know, school district or something or some boundaries which weren't available through Bing or wherever. Could you use that? Oh, absolutely. And then use that as a base map and then through your markers, what else you wanna do using Drupal? Yes, and if you come talk to me after I can show you some cool examples where we've done it. Thanks. Yeah. Maybe we should. Yeah, sure, so Mapbox is really good for like base layers and static overlays and that sort of thing. And like when you're talking about it with Drupal specifically the open layers module, you can import Mapbox layers on top of our into that just like you can like Google or open stream map layers as well. Leaflet also handles that. I would also suggest checking out a project called a CardoDB, which is a very similar stack to Mapbox but it's live rendering versus having to pre-render things. There's not fantastic Drupal integration support with it right now, but it's a really interesting solution for doing more dynamic type of mapping where you need to do like filtering and that sort of thing. Yeah, can I add one small thing? I believe the way it works with Mapbox or one way it can work is that you can create your own canvas and you can publish that under a particular code. Well there's a little module called Leaflet Mapbox. If you put that code into the dialogue there then you can use for instance a starter kit that I showed with that backdrop, with that canvas and it will superimpose the marks correctly on that canvas. I work in the public sector. A lot of governments, federal, state and local are using Drupal on their websites. City of Austin has a Drupal site now but a lot of people working in government especially at municipal and local level are using ArcGIS. I'm kind of curious, maybe some relative comparisons between where ArcGIS is and what the Drupal offering is right now and what would need to occur with sort of some of the milestones or thresholds that need to be reached with Drupal development to where it's almost on par where we might begin to see some local and county governments making that switch over. It depends on what you're using Arc for. If you're using it to do like actual, like more traditional GIS work, like building, buffering maps, doing like the analysis stuff, I don't think Drupal will ever be a replacement for that. I think really the competition is between QuantumGIS and ArcGIS and they're kind of equivalent. I think Drupal, where Drupal might shine is kind of more replacing not so much. I don't think Drupal will ever be something to replace a desktop GIS. Where it might be going in the future is kind of as a data repository. And I could see the possibility at some point in the future of using Drupal as your database of record for storing geospatial data where then you might have QuantumGIS or ArcGIS plug into that. But they are kind of slightly separate domains, I would say, does that make sense? Can I just, I'm not super familiar with ArcGIS but I know recently in the last 12 months they've released a JavaScript API. They have a server stack, I believe. And the JavaScript API there is definitely a lot of overlap with the stuff that is being done with like leaflet or open layers and their JavaScript API. And also they have a rastering component that would compete with Mapbox as well. So without, I haven't actually played with myself, I've just seen some other systems built with the ArcGIS JavaScript library. And from what I can tell, there's actually ArcGIS has pretty good interoperability with services like Mapbox or Google or even your own custom layers if you wanna go that route. So you could render layers from Drupal or get your data overlay layers from Drupal and throw them on top of an ArcGIS map using something like OpenLayers or the ArcGIS library, I think, if anyone knows more about it. Bye bye. That's true? We have a true. And just to throw one other point under that, I mean, one of the things that we see a lot of our proprietary systems in government being replaced by open source solutions in terms of making data more available to citizens. And that's really one of the areas where this could be of huge impact, which is to say that a lot of governments have access to data stores, either in triple stores or in APIs or databases they've collected over the years or even on paper. And providing a really inexpensive and easily accessible way for citizens to see that data and then become interested in that data and maybe even use that data through an API feed separately from the map is one of the really great benefits of this is showing citizens what their governments are providing and creating for them. And I think that's one of the areas that at least at phase two were really focused on trying to sort of replace enterprise expensive solutions with. I'm creating maps where I've got to interact with several million polygons and have them be filterable and searchable. So the geo hash really sounds interesting to me. So first question is, is there an easy way to use that with OpenLayers? I'm using OpenLayers right now and I'm gonna post-gress back in. Sure, so the setup that I currently is using views.json to output a clustered result. And the only difference from a normal geo-json feed is that the clustered result contains the size of markers. So you would need to consume that data in your OpenLayers JavaScript library and the Drupal module. And then it really depends, like out of the box it should just display all the markers, but without any, it's not aware of the items being clustered because it doesn't know about the extra property that I've added. So you would just alter the OpenLayers, I don't know, behavior to visualize these items. And as far as I know, maybe Paul is more aware of with OpenLayers, you can already, based on attributes, alter the styling. So I think that's like five to 10 hours away of being on your particular website implemented. Okay, and this is kind of related, but right now I was kind of scared to get this huge database of polygons into Drupal. So it's sitting out and being served up by GeoServer. Is that done? Am I doing something wrong? Cause now I also, I like the idea to not put too much into Drupal because Drupal will just get slow. So maybe you should just take the ideas that we have created and probably have a look at Patrick's solution because his is not that tightly integrated into the Drupal stack. And because the algorithm itself is rather simple. Yeah. I mean, if you don't need to bring it into Drupal, there's no particular compelling reason to do so. I mean, you can probably index those things. The key is to just get them indexed into Solar. And so indexing into Solar doesn't necessarily have to go through Drupal. And I think for you, the tricky, I think there's probably two tricky things for doing what you want to do. And this is actually a problem I'd be really interested in because it's tricky. So with a point, you know, there's one, there's only one geohash at a particular level that represents that point. But for lines and polygons at a particular geohash resolution, that line or polygon is gonna go through a variety of geohashes at that detail level. So the algorithm that I showed for binning things would have to be modified slightly to kind of be a multi-field where each is actually a set of multi-field geohashes. And that multi-field would be all the geohashes that the line or polygon go through. And then the other thing probably you'll run into is displaying, I mean, even if you want to display a thousand or a hundred polygons, that's a lot of data to serve. So you might want to think about pre-simplifying some of those polygons. So when they're stored into Solar, or stored into whatever intermediary database you're querying from, run them through a simplification algorithm so that when they're actually displayed to the user, I mean, if it's only this big on the map, you don't need to have, if the original source polygon has 10,000 points, but it's only tiny in this tiny little corner of the map, you don't need to display all 10,000 points, you could simplify that down to 100 points. And only when they zoom in would they actually see the full high-res shape of that polygon. So I think there's kind of two interesting problems there. Yeah, right now I'm doing whatever, the ST centroid, and then just dealing with that for finding things and then shipping back the polygons. And they're all pretty simple polygons, they're just real estate parcels. So yeah, I'm fighting that. That sounds like a reasonable solution. And that's what we did with the demo here, is these are actually, a lot of these, see these are boxes here, you can see. And so these are actually polygons, just box polygons. But the only thing we're indexing here is the centroid. And so, I mean, there's some edge cases where something should technically appear in the map, but it doesn't. But generally I think most users won't notice the difference between indexing the whole thing and indexing just the centroid. For most use cases I think it's probably good enough to just work with the centroid. And I'm gonna be here for the sprint and I was just wondering if I'm gonna be doing something general or if I'd be lucky enough to have one of you guys that's gonna be doing some sprints or being around for the sprints to do more map-specific stuff. Yeah, I personally would love to. I'm here till Tuesday actually, so we'll be sprinting from Friday to Sunday. But I don't know in which way we're gonna sprint, so. Just see who's there and then, same for me. Great, thanks. Yeah, so if there are any more questions, I really appreciate you guys coming to watch us talk and all that. And you know, feel free to hit any of us up afterwards either in person or online. You know, we're all pretty Google-able, so. Otherwise, thank you guys. Go team mapping. Woo!