 All right, am I on I do apologize. This is going to be much more boring than the last couple talks But they didn't tell you there there's actually a bed backstage if it gets that bad Smart okay, well, um, I mean already introduced me, but I'm Andy I live across the river in Cambridge Really into the local mapping scene. I have this Bostonography blog that she mentioned that's kind of just for fun, but It's Tim Wallace who works for the New York Times and me We we enjoy making a lot of maps of the Boston area and just kind of exploring geography that way Also, I just want to plug I help organize this map time Boston if anybody's interested in learning mapping technology Who was that See if there's a map time chapter in your city or if you're in this city, please join us. It's really good But that stuff's all on the side most of mostly what I do is I work with access maps or a small company We do interactive web maps basically a lot of custom designed Things for a lot of purposes. This is just a little Example of some stuff that we do and Really what I want to talk about today is Like this napkin sketch is kind of I don't know if you can read it, but sort of a version of of a mock-up of a lot of our projects where We're not really Designing just a map we're designing and building this thing where like a client is going to throw Who knows how many different data sets at it and our map is going to have to support all of that and there's a lot of challenges around that which is What I'm going to get into today When we went to cartography school We we really learned this this mapping craft as an attention to detail and Really finding the particular story that lies in the data that you have and crafting your map and your story around that and really perfecting it based on that and In the I think the reality of a lot of modern web cartography is you can't really do that You have to make this map out of something that you know you kind of know about the data But you can't see the data and you don't necessarily know exactly what the perfect story is in there And you're you're building something that Can work okay, and this comes up a lot at least for us because You know sometimes it's that thing where there's a whole bunch of data sets And it's kind of an exploration tool Sometimes we have to do this work simply because the data isn't ready yet That the client wants us to use and but we still need to go ahead and start working on our thing Or maybe because it's going to change in the future They might add something remove something and of course We don't want clients coming back to us a year later Complaining because the map looks like garbage because they threw this new data set at it so Really what what I'm searching for here is various compromises in a number of areas of Where we find these challenges in our experience and maybe some others experience of trying to balance that really Good design with kind of not knowing how to do good design because we don't know what we're designing And I have here what follows is kind of It's a little bit of a laundry list I'm a little bit me thinking out loud about what are just trying to go through some areas where we find some challenges and Think about what the problems are and try to identify certain solutions Which are never perfect again. This is about compromises and Try to find you know some things that other folks have have done in these areas, but also a lot of this I'm just trying to Learn myself, so I'm very interested in other people's experiences and and so forth and I should caution I don't have tons of answers a lot of this stuff is just questions and saying well This is hard. I don't know what to do And so if anybody does have answers for some of this stuff that would be awesome So I hopefully this isn't like too disjointed in my in my ordering of all this stuff But I'll go ahead and start With my first category here, and I think this one comes up a lot for us Just data classification and I mean in particular for Coropleth maps and how do we you know divide been been our data into different classes? And those are going to be the different colors on the map I don't know if you can see this very well, but I guess I could try the zoom thing So this is sort of what I'm talking about different ways to you know arrange your your data classes on a map and this is a graphic from John Nelson who wrote this really helpful article explaining some different classification methods and how they affect the map in the end and If you're not familiar with John Nelson's work Just ignore me for the next 40 minutes and go to his website because it's really cool There's a post here. Oh, and by the way, I should mention There's a link to this these slides in the first page. I also just tweeted it. Hopefully it went out if you want to Actually follow some of these links here and and see that article But this is just a really simple example of what a difference This classification scheme can make this, you know, these three maps have the same exact data but they're just chopped up in different ways and you get these vastly different maps and If we are designing some system of interactive maps That has to accept a whole bunch of different data We don't want something that looks like this one popping up very often because it's not a useful map so We want some kind of compromise that's going to work reasonably okay for almost anything so This frustrates us a lot, maybe we're doing it wrong, but we do these designs in my company a lot where We kind of design around this idea of this really nice almost normal ish Data distribution makes a nice histogram and all this stuff, but nope It's like never like that or there's always one or two with you know Some stupid outlier down here that just ruins everything Pretty much every time there's always going to be one So yeah rule number one just that will happen always so what I'm Yeah, well, this is what I'm getting at we need to have some kind of classification Rules that are going to work pretty well for almost anything without us actually having to look at the data and figure it out For that data set and there's a couple of common ways That cartographers do this which I'll get to in a second But these are some important questions to consider on how you've been your data. This really goes for any map And it's not just For this this particular purpose, but we're trying to make a compromise It's going to be a pretty good answer for all of these so first and foremost is the map going to work Is it going to look good will it be useful? You know like that map a minute ago that was all one color basically that's not a map that's useful And it also doesn't look good Same I just wanted to throw charts in here too because Interactive mapping often involves also other visualization elements like charts that are accompanying it. So same question for those Are the brakes meaningful these class breaks the you know the numbers between the where the bins lie By meaningful, I mean does it actually have some sort of real-world meaning this number that we're using a good example? I think that came up a lot in the cartography classes and colleges if you're mapping income For instance, you might want to have a break. That's at the poverty line or at certain other meaningful area levels Or if you have a diverging set, maybe the number zero is a good place to have a break And are they understandable meaning by this I mean Does the reader to the map user Get like how these numbers were derived and why they are why they are where they are So a couple slides back one of John Nelson's examples was a standard deviation for example Classification where you're making breaks at like the mean and then one standard deviation out and so on But depending on your audience I can imagine that People may not understand what that is and why why you chose chose these numbers and what is a standard deviation? What does it mean? And then this sounds trivial, but are the are they nice numbers like you don't want You want four hundred you don't want four hundred and two point three seven eight five or whatever unless that's a meaningful number so these are some things I think that we're often trying to compromise to Do okay at and a couple of just I just I'm gonna name two like common methods Which should be familiar to anybody who does mapping but these optimal breaks breaks Developed by George Jinx in the 70s. I guess It tries to find these kind of natural breaks in the data by this algorithm that maximizes similarity within groups and difference between groups If you're a JavaScript person check out atomic rights statistics library has an implementation of this you could look at But of course has downsides like this is going to give you some of those weird numbers and And they're gonna be crazy different for every distribution and it may be difficult to compare Or to kind of maintain consistency This is the kind of a fake classification, but it might look something like this on this histogram You know you for one thing this outlier is definitely going to be all by itself because there's clearly like a natural break in there And then you would have some other sort of reasonable breaks in the middle Quantiles then is another kind of easy go-to where you just simply divide your your data into equal sized bins On the map, but I guess I don't have a good picture of this, but again, there was one a few slides ago It kind of guarantees that your map is going to have some variety. You're never going to get that one color map You're gonna see an equal number of Entities in each color So can give you a pattern which it's not necessarily truthful, but it's there Just a code example d3 as quantile scales and by the way, I have like miniature code things thrown about here It's mostly just JavaScript often d3. That's what we use a lot, but this isn't a very technical talk all together But there's big downsides to this like You get these groups that just don't really make sense because you might have numbers really far apart from each other thrown into The same group might have two numbers right next to each other split into different groups So that's a big downside to these So here's an example if you just did what quartiles on this distribution It's you know looks somewhat reasonable in here, but then you get this thing where This doesn't really belong with all look like that So not perfect, but something we've done a few times is Use percentiles, but not even even buckets like this actually kind of try to pull off a high end maybe a low end into its own class to Fix that problem a little bit not really I Don't know if you can see my extra color here, but you know now this guy is still joined with these But it's only joined with a couple of them. It's not quite as bad and Oh, I guess I should mention. Yeah, one of the problems with percentiles or quantiles is you are no longer You're not really looking at the actual numbers anymore necessarily and even my little scale here This is not giving you actual numerical values. It's just saying 50th percentile and you don't know what that is What we often do in a case like this is we'll label our graph Maybe our legend this way with just 25th 50th something like that, but then relegate the actual number to an interaction maybe you hover over this to over this thing you can see This of course, I'm not interactive But imagine it if you hovered over this thing you would see some actual value of where you are in that histogram Or on the map, of course Well, this is just a screenshot of something we did do like that. It's a map of some health data in Public health data in Illinois and Very much like that. There's this one outlier way out here, which is Cook County, Chicago And as Ken Field is in the audience, I want to assure him that this is not because there's a lot of people in Chicago This is actually a normalized map So it's not a bad choropleth just whatever I don't remember what the variable is but it's it's It's something per capita. I think Anyhow So anyway, that's just an example and one final thing that I've done a few times is Kind of getting away from that understandable Criterion but rather than dividing up the actual data itself to do percentiles based on the unique values So transform that the problem that we often encounter if we don't do that is you'll get these data sets were like Half of them are just the value zero or something and technically if you did like quartiles on that then your zeros would span like two or three of the classes Well, I guess if it's half it would be two classes of that data and you get this really weird map where like you have zero labeled on two different colors and Probably the code is just gonna end up coloring them all the same. It's really confusing so this is something I've done sometimes is Pull out the unique values and based the classification scheme only on the unique values and not on all the values Simple little code code example there in JavaScript. There's probably easier ways to do it Okay That's my last bit on data classification The next bit I wanted to get into is simply how Things get really messy when you don't know what's coming you end up with stuff on top of other stuff and it's illegible So this is probably pretty obvious, but things are always going to overlap and Really thing a thing not to forget is just to if you have say circle symbols Make sure the small ones are on top of the big ones. Otherwise you can't Get at the small ones or you can't see them simple little example here bunch of circles all on top of each other, but They're ordered right so you should be able to like get your mouse on every single one of these Also overlap just a quick thing goes for not just the map, but also like labels and things like that histogram here where by default it would just label everything directly under where it is, but We wrote some code to make sure that things get spaced out and never end up on top of each other then there's this there's issue of scale and Good example of this is a county map Where some counties are huge and you can easily get at them and some are going to be really tiny and you can't unless you zoom Or maybe you don't even know what your geography all of your geographies are and You don't know if they're going to be big or small or what? This this here is a kind of a fake mock-up using a New York Times election map But it's trying to show a Thing that we have done from time to time and New York Times Sometimes also does some similar map navigation, but it's your basic kind of slippy map You have these pan and zoom Capabilities you got those zoom buttons you can freely move around the map you can zoom into wherever you want But also we add this kind of extra More design Guidance of the map user by not forcing them or not relying totally on them clicking zoom button every every time but when you're zoomed so in this example you're zoom when you're out at looking at the whole nation We have you if you click on a state it zooms right into the state rather than Interacting with that county level and then once you're zoomed in a little bit then you can interact with counties You can still pan and zoom or what? and it's We've done this a couple times just to in an attempt to Give the yeah give the user a little more guidance and try to Design what we're doing so we don't know everything that's going to be underneath there But we do know that we have say states or maybe it's countries or something and you can kind of take advantage of that And use what you do what you do know to try to guide people into the more unknown scale This is to say nothing of actually varying extents Like if you don't know if your data is going to be in the US or Asia or what I that's another issue. I suppose to Which I apparently don't have anything to say about No, I mentioned it Okay, then there's this next bit No Is to just to remind The cartographer that there's probably always going to be holes in the data missing data Maybe bad data and you just don't know and you always need to be aware of that always catch that those those null values and explicitly designed for them because they are in the sense they are data as well and those of you here last year Andy Kirk gave this this nice talk about the design of nothing and One of the sections that of his talk was was really about that. It was about how do you about designing? for the lack of data and actually what a what a powerful element that can be there's there's certain visualizations where that he showed where these gaps are really meaningful and they're kind of telling a story and He showed my there's a bunch of examples there You can check out check out the video of his talk last year because that's pretty good So the first thing we always have to remember is that this no data can come in a variety of ways and We at least need to know what we might be looking for and hopefully we do But when it comes into that when it gets piped into the JavaScript, however, it gets in there You know it might come up as null or undefined or not a number or a blank string or an actual like literally the word null Or some you know stupid number that they put in to stand for no data So yeah, the important thing is to just I guess I don't really have a great example of this but There's a lot of functions we write where of course the first line of this if this function is like if this thing's not Or whatever then do something else Because if you don't catch that stuff the map can get confusing maybe things might just disappear or You're gonna get some default thing like an SVG solid black fill which looks terrible Things like that so you we really want to be explicit about it and and not let the map get confusing because we want we don't want the map reader to confuse no data with data Just a little thing that I always forget to do is remember that zero is different from like null and there's some a Code thing you can kind of get tripped up on if you just say if this doesn't exist It's also gonna think that zero doesn't exist, but usually zero is actually data that we want to catch So a couple of examples of how you can design for null data and again You should just go back to Andy Kirk's talk from last year because there's a lot of this but Texture is something we've been getting into lately. This is a Modified example of this textures JS map, which is really cool I think it's for D3 and it makes it really easy to make these SVG textures and That is a lot better than trying to figure out how to write a path command for a stupid diagonal line and like It's been figured out for you. You can make nice. Maybe hatch stripe patterns to indicate your no data It's very clearly distinct from colors You know, we also just color things gray a lot if it's no data But depending on the color scheme of the map or everything else that could get lost could get confusing so check out this textures thing if you're a D3 person it's pretty pretty great These lines are just one example. There was all kinds of patterns that are built right into it Just another specific example is a time series data and being explicit about when there's gaps in there and not trying to interpolate and suggest that there's actually data in between two points. So this is a You may recognize us from the iOS 8 health app that somehow figures out how many stairs you walk up, I guess But yeah, so there's all these days here where I guess I didn't walk up any stairs and I I do live on the second floor so Apparently I didn't leave the house But I like how it really makes this dashed line the shading underneath it to make it really clear that this is not Like a real trend. There's just nothing in between But well still kind of maintaining that That consistent line. I guess I updated this yesterday because the previous graphic I had in there was like Average of one and there's like a bunch of days missing lazy month. I guess This is this is a screenshot from an old old map. We did some years ago. It's probably a little hard to see but We had a I guess this was an animated map of proportional symbols showing Jewish population in various US cities Over some time period and it would do it what you expected You know the symbols would grow as the population got bigger, but we had all these years Where there was missing data for certain cities where so there's a st. Louis here We had data for 1850 But we didn't have data for 1890 and then maybe we had data for 1910 or something and We attempted to do Something here. I don't know how well this really works, but we didn't want the symbol to just go away We also didn't want it to remain To say 600 because we know it's not 600 still in 1890. We just don't know what it is so what we tried to do is retain the last known symbol But give it this like empty. They're all pretty small but empty circle character and Hope that that was clear. I I don't even remember where to find this map anymore There was probably like a legend with it that maybe indicated that But this is an interesting thing I actually I would like to hear other thoughts on how to Kind of design for gaps in animated maps where you are missing data Okay, next thing a little bit about text and I don't have a lot here but I wanted to bring it up because I'm talking about maps, but It's easy to forget that a big part of maps is actually the text that goes with it Whether it's on top of the map or beside the map or whatever that's It's almost always there Marty Elmer wrote this nice post a year or two ago about Pros in cartography and a lot of good examples. That's what this link is up here. Don't read the rest of it but I Only want to mention one thing this often happens to us is that we have this nice designing We're kind of assuming there's going to be this nice little text snippet here somewhere, but What actually happens is sometimes? Yeah, we get a few lines of text But sometimes the data that's sent to us is like this 4 million word thing and if we're not careful our text box is suddenly going to like get huge and just cover up the whole map and whatnot so just You have to be mindful of Some CSS properties like you know restrict the height and make the user scroll through the box If it's going to be really big Otherwise it can kind of ruin the layout of the map And certainly I'm sure there are plenty of other ways to account for things like that like you know Cutting off the text and having a like read more button or pages or something like that Something to be aware of anyway Quick thing here. I found this example of one of our projects to but Abbreviation for I'm talking more or now about like label size things Here are the labels on the bottom of this bar chart Kind of get restricted in size and this is just due to some constraints of our layout So we kind of abbreviate truncate them with an ellipses and when you mouse over it, you can get the full name But I just wanted to point out how you that you can be more You know thoughtful about how you abbreviate words, you know here the names you just kind of cut them off at the end, but this is a congressional district and If we abbreviated every congressional district the same way that we did these names, they'd all say the same thing They just be like co n dot dot dot and so remembering in this case that the number of the district is actually something That you need to see we abbreviate a different way we cut off the word and then show the number So this then this is an interactive thing by the way You would the chart doesn't always look like this You could have like five congressional districts in there or you could have just like one bar where the thing actually does fit So we needed to be able to account for all of that Another thing that always gets me is remembering to format for singular Words like we have these labels all the time where it's like x number of Whatever's and then you forget that if it's only one the word is going to look different Also over here orders of magnitude this label, you know, you don't want something that says a thousand thousand you want that to say one million so you need to Be smart about formatting things like that and by the way, I guess I don't mention explicitly, but If you're a D3 user, you know that the axis Thing there and the scale ticks are really amazing because they give you these nice pretty numbers out of any range But they're not really going to label them like that for you That's just a little like, you know simple code for how you might do that. I do this quite a lot say Add add an s if it's one if not don't add anything Okay, so a quick couple things about text there and then kind of stepping back to the map as a whole This term the lorem ipsum map which I'll reference in the second but I'm thinking about how the The whole gist of this is that when we're designing building a map, we don't have real data We don't necessarily even have realistic data So we're using some kind of stand-in whether it's something we made up or something that's close to what we need or what and We're building our whole thing around that And that can be a problem sometimes. There's this little Usability study by Rob Roth and Mark Harrow were that kind of cautioned against really designing all your interface without Also designing your map and knowing what it's going to look like because the user experience can kind of be degraded So this this little study was based around this map which we did Back in grad school at the University of Wisconsin Also go badgers tonight We so we ended up this the map itself is kind of this like soft fuzzy green blue thing But we didn't really have that design Finalized really well. We went to town building this interface. So we just started building this like Cold metallic looking UI because when you did things in flash in 2007, that's what you did and So there's a I mean there ended up they did this little user study afterwards a usability study and There was some negative reaction to that. It's just the How incongruous this this map and the UI were so That's one of those shruggy moments I don't have like an answer for how you do that if you don't actually Have the material if you're a map, but it's something to keep in mind that said If you're making fake data just to test your stuff out and to build your UI around it It actually could be a really good test of code because this is a super unrealistic election map in Egypt Egypt that we're kind of in progress on but if if your map can work like functionally technically if it can work with this Terrible data then it can probably work with real data So in some sense is the the worst your fake data the better So this thing like you have precincts that are like these stupid multi polygons and stuff like that and the map is okay, so We're good Caroline fish was was kind enough to get a thing posted this morning so I could link to it on in here But if you go to this you should read her post. It's she's talking about an experience that she had Where she's using open data open street maps specifically while designing some map templates To stand in for what in the end was going to be like the total opposite of that Classified defense data that she could never even see and There's some challenge obviously some challenges around that like getting this one type of data to stand in for another type of data When you're designing it and you know a little bit about what it's going to look like, but you don't know everything I Like her term trying to come up with smart dummy data so you can look at her post is also talks about getting into open street map and So starting to wind down here. This is all kind of getting at Trying to approach that more traditional human driven design where you're really paying attention to details But doing it somehow by rules and code I have a quote here from something from Daniel Huffman a cartographer who I Love among other reasons because he's a real rise of the machine's alarmist and cartography Like he might be a time traveler from some carto apocalyptic future So I want to do a little just the sort of thought experiment here This is a map that was around the internet so a couple years ago this wall map reference map the US by David I miss it's like heralded as the greatest paper map ever and it's it's a really it's a good example of a modern Example of that really human human design map like yeah, it's made on a computer, but So just to kind of extrapolate this a little bit because these days we're talking about these tiled web maps of the whole world Apparently this map it's one to four million scale and it was apparently about 6,000 hours of work So if we take that out, it's about zoom level seven for six hours Have that down to zero Double it up to say 18 That's being conservative because map size actually quadruples if you zoom in one one level So this map is going to take him 2,800 years to design with that level of care and 222 days And that's only the US so this is just a point to what I want to call big design this challenge of making good design of Data that is just far too massive for you to really inspect as a really caring human that's the question there and Pretty specifically. I'm talking about designing these global maps And we're usually using open street map unless you're Google or somebody, but this is open vis so we're talking about open street map and this is a data set that's got millions of users and You just you can't guarantee there are conventions But you can't guarantee that the quality and consistency of this data is going to be the same everywhere across the world And that your design is going to work on every little bit of this data So your challenge though is to come up with these rules that it are going to work even though you could never See it all So I just want to summarize something from Nikki of map box at nasus last year She talks a bit about this look quite a lot about this actually because they do this kind of work, of course Just some rules she she suggested prioritized prioritized your use case areas in your specific use case of course and Prioritized the most common examples of something But then you know do go out and look for some atypical examples and try to develop your compromises off of that and their software map box studio now has This kind of feature here where you can view multiple areas at once you can kind of inspect what? Certain feature might look like in different areas of the world I don't we have map box people here because I don't I can't quite figure out how to use this and I could use some help but That's an example of some tools I can help with that Also some of the open street map API's and so forth Where you can kind of inspect details about types of features and tags and geographic distribution and actually get the data and all this stuff This is an example of some highway tag that As you can see it gets used a lot in Europe But not so much in the US and you could easily be fooled into thinking that this tag just doesn't matter Don't design for it if you're just doing a map of the US, but You don't want to do that if you're doing the whole world you need to be aware of such issues and you can explore them through something like this and Of course with open street map and I glossing over open street map here I hope most people know what it is, but if you don't maybe you don't if you're not a mapper But it is this just massive open community driven geographic data set so if You find something wrong in the course of your inspections that you are doing of your design you can just go in and fix it and It's fixed So if you're a mapper definitely get into open street map use the data first of all, but contribute to the data There's a whole community out there, which I am not as into as I should be but it's there so I Think on that mini plug for something super open. I'll I'll leave it there, but Thanks, I look forward to