 Hi everyone, my name is Sameer. So before I jumped in, I hadn't come for the previous meetings. I wasn't sure how to position this talk. So what I've done is, okay, first of all, how many of you have worked with D3 before? Anyone? Okay. And back then, a lot of people? Okay. Okay, so what I've done is that my presentation is going to talk a little bit about the intro level of D3 and back born and the kind of places where one struggles normally. And after that it becomes a breeze. So I've taken simple examples and I've worked them through and I have examples on JS Priddle. So if you have your laptops that you want to experiment, you can open that up and you can try out on JS Priddle. I've uploaded this presentation to SameerCycle.github.com So you can download it. Those are links to the JS Priddle. And if you want to experiment, we can go inside this end. Okay, so I have just a quick introduction about who I am. So I had this company called R2. What we tried to do is design and pick up the thing using mobile and phone for companies that work with people. So this could be in the areas of financial inclusion, healthcare, energy and the like. I won't plug in anymore. I just wanted to give you some context. And this is the kind of stack that I worked on. So we worked on a whole lot of things from CouchDB to front-end JavaScript frameworks and things like that. So it's really interesting that every time you search for JavaScript you find a new framework and someone else was using it in interesting ways. And so while preparing for this presentation I just googled what's up with JavaScript trying to figure out if there was some sort of science behind all of this explosion. Of course there are no JS and all these other big frameworks. But one interesting thing came up which I hadn't read before and which is this AdWords law. Have you guys heard of this before? He says any application that can be written in JavaScript will eventually be written in JavaScript. And that's so interesting because just yesterday I was looking at the entire architecture that we've been working on. I realized we have so many different technologies but all of them could be compressed into some sort of JavaScript framework. And eventually we'll start working on that. So D3 is basically data-driven documents for short. It's a really powerful library that can take data mostly in JSON format and you can do a whole lot of things with it. So it's written by, I'm forgetting his name, but it's an amazing library and he's done some amazing, he's played some amazing examples. But the amazing part about this library is not just a charting library. So if you work with Google visualization, or any other of those visualization libraries, they all have some structure on to you. So they say this is the definition of how a chart looks. Your data needs to look like this. So x, y, z, some column name, etc. This is the format. Whereas D3 has gone the other way around. D3 has said this is how your data looks and this is what you want to do with your data. And that's a pretty interesting thing because then you can create any kind of visualization. You can use all the coolest things about HTML5, the DOM elements, etc. to create visualization. So this is extremely powerful. And I can just show you a quick example of some of these things. So this is D3.js.org. They have about 30 maybe more visualizations. Some of the interesting ones, this is a simple one, but it's really interesting. So I'm using my keyboard to change the year and this is showing some of the population, the number of people who are in these 8 objects versus those 9 people in these. So this is a simple sort of dynamic chart that he's made. Another interesting thing is you can have this Coropet, like the graph, where he's populated unemployment data on this map. So he's just taking a JSON feed and he's overlaying it on the map. It also creates some stupid and silly things like this. So this is all written in pure JavaScript. This is no flash, nothing else. And the way he's drawn it is very interesting. This is an ellipse and he has CSS transitions for that stroke. And each one is just moving randomly. So it's very interesting what you can do. Some sort of more interesting commercial examples. So you can have transitions, you can have all these colors and graphs written. The reason why we like to create custom visualizations and charts for our clients and the users. So one of our 8 turns built simple prototypes. I'll just make some mentions. So suppose I wanted to compare the performance of various branches across Bangalore and I wanted to have an ability to move my mouse around and see and highlight my particular branch and pick the differences. And if the data behind this keeps changing, you'll have to move back and forth. So again, D3 allows this. This is really interesting. Another simple chart that we built was using scatterplot. So nobody really allows you to build a scatterplot where you specify, you know, x, y, z. Make that x, y, z change. So there's a Google Time Motion Chart but we wanted something which was not dependent on years. So if the data changes, these balls can move on different parts of the axis. So D3 gives you really raw APIs where you can change, you can put in SVGs, you can change development, all the properties, everything that the DOM has. So let's sort of look at how... I'll just show a quick background about Backbone. So most of you know Backbone has got as a MVC, a minimalistic MVC judgment framework. And what I really like about Backbone is its abilities and its definition of views and how you can fetch sync data and events. So you can specify if you want your data to come from CouchDB or if you want your data to come from a particular place. That's really interesting. And what I don't like is the ability, you know, so something like Knockout presents the actual data binding feature which is missing in Backbone but Backbone, I think, is a lot more powerful. Anyway, so if you guys are interested, you can follow this example that I guess we do have created. But to look at the syntax, so what I've done is I just got some support here and I'm just showing you, you know, a very simple way when step by step. So V3 has a very jQuery-like sort of syntax where you start by, you know, specifying a selector. So in this case, I'm just selecting the body and I'm selecting all the elements in that. I may not have any elements. And then I can specify data. So in this case, the data is very simple. It's a simple array. And then I say enter, right? So V3 is designed from the point of view of stages. So there are three stages. Enter, update, and exit. That is when you add new data, that's the enter stage. When that data changes, that's the update stage. And when that data point drops out, that's the exit stage, right? So you can specify animation, you can specify what needs to happen in each of these things. I've just written a simple thing saying that this array, when you enter it, you know, when these are new points, create a P element, pick text in that which looks like this, right? So very simple. And it's just three lines of code and you can get this, right? Suppose now let's move it to the next level, the exit, right? So if you remember, I had six data points here, 15, 16, I'll remove 16. And so my graph looks like this. And I've said, you know, on exit, change the style decoration to line through. But can you see the mistake here? I removed 16, but something else got crossed out, right? 42, because it just assumed that the length of the array is lesser and therefore the last point has gone up, right? So we need to tell a D3 how to understand what's the unique data element, right? So we can specify, when we specify the data part, we can also specify a function which takes B comma I, so B is one of these elements and that is the index when it comes. And I return D. So in this case, my values are not changing. I'm just, you know, dropping that particular element. So now you see 16 is missing and it crosses 16 now, right? So this is just the basic framework for you to do anything. You can create charts with this, you can, you know, just stylize your page better and you can connect this to WebSockets or anything like that, right? Very simple. So let's focus a little bit more on charts. So now, you know, when we're plotting a graph, right? You have X comma Y, that's how you plot it. So for each of those functions, you can create a scale. So basically it is the input domain from having it to the output range, right? So I'm seeing create a linear scale in 0 and the max of data, right? And the ranges, these are some pixel values on your graph. And similarly, the Y scale looks like this, right? So ordinarily in the sense that over here you could even have text, you know, suppose in the first graph that I showed you, I had different branch names. That's discrete data, that's not continuous data, right? So I could have anything over here. Specified my X and Y axis. So let's keep that aside. Let's start with actually plotting the graph. What I'm going to do is I select the body. In that, I create an SVG element and I specify attributes that I want. So the width and the height of my graph. So this gets created, right? If you have questions, if you want me to explain more, you know, just feel free to jump in. Have you guys worked with SVGs? It's a simple, it's a new DOM element, right? It's got more flexibility. So earlier, what we would do is to simply draw, like, tangles and things like that. Now, using SVGs, you get better animation, moves really smoothly, and there are amazing properties that you can know of some of them until I provide for this presentation, right? So another interesting thing that we've done is that so the body is the body of the HTML body, right? SVG is the container of the graph. Now, in SVG, we will create trip times, which will become a bar graph, right? Now, if I want to transform my graph, if I want to, like, rotate it, give it a 3D effect, if I want to do anything that needs to apply to all rectangles in it, I can create a group element called G, okay? So this is an actual SVG DOM element called G, right? And I can create some transformation. So this I'm just simply translating it, right? And so if you can see the HTML, that's what it's getting created in the background. We mentioned the three stages, right? Enter, data, exit. Now what we're doing is we're saying in the chart. So the chart was this, you know, this G container, right? That's a chart now. And inside that, we're saying all the rectangles, okay? When I start, I don't have any rectangles in it, but we have selected all the rectangles. To that, we apply the data, okay? Which is a simple area at the moment. And then we say every time you get a new data point, you create a new rectangle with whose y attribute, this is the y attribute, right? Whose width is specified by the domain functions we created. Remember, x and y are domain functions. Whose height is, this is like a range band. So we created, this is like a simple band, like 30, 40 pixels or whatever, or 20 pixels in this case. And the text that you want to put over here is simply the value, right? So just look at it as, you know, it's like a chain, you're chaining all these calls. And by the time you're just done with this block, you will have a series of these statements, right? Which will start looking like this, right? If you would like to see this in business, I can't change this thing. So just so that we, I mean, we can go over this again. This is here. So this is where we specified our axes, right? Not the axes, so we made these scale. And this is where we specified our chart, you know, the object. And now we've created the rectangles, right? And so if I stop here, you would have got just a simple graph. See over here, we wrote, you know, we returned the text, so the text actually shows up over here as 48, right? But it doesn't really show up over here because the color itself is highlighted, right? So what we can do is we can also add a text element. So again, SVG is also our normal element for text. And again, we specified similarly, you know, this is the, you know, this is where you need to place it around the x-axis and the y-hide, right? So this function is like a repeating, it's a pretty interesting thing. If you, you know, jQuery you've done dot map, right? Or dot each, right? Where you can specify an anonymous function where you will get an item and an index of it. So this is very similar to that. So you get d comma i, you can pick the variable that you want and these are just padding, et cetera. And this is just specifying that it needs to be aligned to the end, right? So I get 48, 15, and looks like this. So these are the text elements that have been added. If I want the lines, I can also add the lines. So another cool feature is that x dot takes, right? So text will just give you 10 divisions, you know, in that continuous range, right? And you can do that. So it looks like this. I mean, this is still not that much fun, right? Good. And if you want to, okay, I'll come back to this. Okay, so now we've got a simple graph. It's a static graph, right? We've written how many of us, 20, 25 lines of code, right? But it's a static graph. The next interesting thing that you can do is when you want to update these graphs, right? So the good part about using D3 is that it automatically sends if your array has changed and what it needs to do, right? You just need to simply collect that data. It might come from the agile. It might come from the WebSocket. I need to give it back to that object. So in the update phase, what we're saying is taking all the rectinals, a static transition whose duration should be one second, and in this transition, you change the width, right? Which is this width. And let the height remain the same, right? So what happens is when the data changes, the graph goes back and forth, right? If you remember the thing. But what has happened is we've left the text behind. We're just playing with these like angles. They're going back and forth and we left up text behind. So what we need to do is we need to apply the same update logic even for the text, right? So that is simply showing you how you can do that. A cool feature that you need to remember over here is you only need to specify the variables that have changed, the properties that have changed. So you don't really need to define the whole thing that, you know, the height should be someone else, the padding should be someone else, the colors, yet they won't change. Whatever you specify during this transition, then you can update it, right? So now the text also moves. We can see that in this case. So I've just simply done a randomized function, if you can see this. The bar moves within one second and the text moves within 1.1 seconds. So it gives it a slight interpolation feel like initially it's coming to start and then it closes in really quickly. So these are interesting things that you can play with. You can even change the opacity and things like that. So everything that we used to do with CSS or jQuery, etc., all of those properties can be applied here. You can chain all of these cores, right? Isn't that making sense? So the interesting thing that one takes time to understand is this enter, exit, update phase. If you look at it as stages, you understand that it just needs to happen, you know, you can really master a jQuery. After that, it's just like an API. You just find out what properties you want and things like that. Okay, so the next part of how do you connect this to backbone? Yeah. Does it have any high-level constructs? The API is so useful. But when I want to get some fast done, I look for something that I can accomplish faster. And then we look for a way to make it difficult for it. So does this have any feasibility to build that kind of high-level constructs? So I would say, yes and no. In the sense that if you're just looking for a pipeline, if you're just looking for standard visualizations, then I would say go to the cloud. It's like out of the box. You can apply it, it'll work. But suppose you want to build a dashboard, which is what you want a lot more control than what any other graphics like they will give you. So you build this once and then you can reuse it. So it's giving you a lot more, you know, raw access to each of these DOM elements and a lot more control. But once you build it, it becomes a model of package that we can use everywhere. So I would put it this way that if you can't see what you want in any of these other libraries, then you come to this place. And if analytics or dashboard is like a very big feature for you, and it's like the most important thing, then you'll do this. Or the other is if your data is really large, if your data set is huge, then D3 is very good for that. So D3 is a really... So the graph that I showed you here, this Apple versus Amazon versus IBM is plotting. It's just got a JSON file hanging on the server, which is pulling out and it's drawing. And it's happening on the other side. And it's doing all these animations and things like that. So if you have a lot of data, D3 makes a lot more sense. And if you want custom type, so... Okay. So the next part about backhaul, right? So I was mentioning that if you build your custom type, then you want it to go everywhere, right? And that's where backhaul becomes really cool because it gives you this modular approach where you can make a graph object like a bar graph or a scatter plot, etc. And then apply it to various contexts, right? But the hard part is that you don't know when it renders, right? It's only when it's given, right? Whoever's worked with backhaul. So normally when we work with jQuery, we used to write a simple body on-load function and in that we used to do everything. But with backhaul because designed for single-page apps, something might render when an event happens and really it's not a body on-load event, right? So D3 becomes... It becomes hard to tell D3 where to plot, right? Because the page will not get ready or the HTML will not get ready. And that's the real hard part and if you crack that challenge, then it's done, right? So it seems hard when you're starting but it's actually... But anyway, so backhaul typically has sort of two main functions when you initialize the object that you need to do. And when you want to render any of the changes so you can have a simple thing called here I want to put my graph. And when you initialize, you initialize the container element of the graph and when the data changes, you can simply redraw all those points, right? So whatever we did in those steps, one, two, three, you simply dissect it and place it in different parts. The important thing to remember is that your graph is this. Interesting, right? So if you notice this graph that we wrote as a prototype, you can see that the data is changing and the bars are moving all the way. But can you see the axes? They don't look so clean and nice because they get redrawn again and again, right? So what you need to do is you need to really understand whether you need to look at the graph which part of your data changes. So if you see the difference like now that this is becoming darker and darker because it's getting redrawn, right? So you need to segment your graph and look at what needs to be updated constantly and what needs to be drawn only once or what needs to be drawn can have something really drastically changed, right? And so you segment those pieces out and that's when you, I mean, that's how you place it into your backbone view, right? So anything, so any change in data that needs to be drawn needs to happen, yeah. Anything that needs to be drawn only once like axes, chart, directory and things like that, yeah. So, you know, those are things that you put it in, you know, frame or any other function that you want to pull it, right? So what we do is when you initialize the view, right? So this line bind all to this object you need to bind these functions. What this does is that it it shows that the context, right? So in jQuery you use this element about, right? This, this, this, this starts something. In backbone, this is not, I mean, it's used in the same way, but unless, if you don't have this line, it'll get confused, okay? It'll think you're talking about the, like a class. So this would refer to the object rather than the type rather than the object. So what we do is we say bind all this object to these functions. So in these functions, when I use this, I use that particular object, right? And collection is nothing but your series of data points, right? So we say that on bind, when you have a reset function, that is before data is pushed into it, you reset the data, the collection we added, right? So that's when I want to draw the frame of the graph. I want to put the title, et cetera. And then any time the data changes, I want to render that particular row those data points. This is a crux to the problem of backbone and D3. If you have this statement here. So backbone draws at some stage we don't know when, depending on who calls it and things like that. But D3 needs to know exactly where to put its elements. So if you use $this.el where D3 has a place where it can write. And when backbone finally puts a view into the DOM, it takes all of that and the chart is drawn, right? So this is the part which was not very clear when you start off with backbone and D3. The rest of it is the same. Yeah, so that brings me to the end of this. I have these JS produce up. You can experiment play with them. But the real fun part about using D3 and backbone together is that it gives you the kind of visualizations you want, right? Or the kind of events. So like if you're building a dashboard, you want this graph to be connected to this graph to be connected to this graph. Let me show you. There's a brilliant example of, I think, Square had created this. It was a dashboard for, yeah, Square had created this cross filter library. And over here they have, they're pulling data for all the planes that are leaving, you know, the airports and their departures and arrival schedule at a particular time. So now this is a huge amount of data, right? So these are the airports, you know, the departure, the desk incoming, whatever. This would be the destination. This is the origin. And I can change, you know, I can pull this and it'll update the data. Yeah, you can see I'm changing this. So I'm moving this, you can change the data. If I select one of these guys, you know, it's a time update. If I'm taking it between 12 and 10 and 4 p.m., right? Where, whatever, plus or minus 10 or 20. So it's filtering, it's filtering, you know, can you see this part? It's saying almost 200 and, almost 231 data points and it's filtered 17,000 out of it, right? And this is all happening on the browser side. So that's a huge amount of data, that was really not possible to do. So this is where, you know, something like D3 would be really amazing. Thank you so much. Can we be analysis? Sure. So basically the data for this example, usually these three has to get everything and then properly. Or can it be some kind of streaming? Yeah, so even your web softwares, What feature does it make? As in when data comes, you can draw it and we use it with CouchDB. So we use them as CouchArts. So whenever something changes on the background, it again updates it. So it's like a long pull. So I didn't want to get CouchArts because that will again complicate the example. You just want to focus on D3 and iPhone. But yeah, that's where you can support any kind of data. Can this be based on JQuery? Sorry, no, it's not based on JQuery. But it's not, JQuery likes syntax. So basically, can you store JQuery and use it instead? No, that's what I mean. I don't think you should do that. But I think when it comes to simply manipulating the chart objects, you should use D3. And you can use JQuery for whatever else you want. But you could... I won't do it. Does it play nice with boolean toolkits? Sorry? Does it play nice with boolean toolkits? Yeah, something like that. We have a regular version embedded inside of that. That's easy to do. I haven't write that, but I would guess it depends on that toolkit. Does it allow some sort of JavaScript to a snippet and it should work? Yeah, I think that should work. If you can load it on demand, the D3 library should be fine. Bootstrap is fine. We use bootstrap and D3 together. Bootstrap is... I mean, it's mostly on JQuery. But I haven't done anything with GWT and D3. So I would flip it the other way around and say, does that toolkit support snippets or not? So those are public examples that I've shown you. The first three or four. And then the next two, which was a bulked up in a scatterplot...