 Hi. Thank you very much, Adam. I was like looking forward to some sort of a pun and you did not disappoint. So I moved the microphone a little bit further away. Does the sound, is it okay if I talk from here? Good. Still. Cool. Okay. So hi, I'm Shirley again. It's not... dang it. Okay. Let me move my notes back over here. All right. So hi again. I'm Shirley. I'm going to talk about Backbone and D3 and I'm super excited to be here. So thank you very much for having me, BackboneConf. And before I begin, my slides are up online at that link where the bunny is pointing to it. And thanks to the awesomely huge projector, it's about the size of my head. But I encourage you to, if you can get Wi-Fi in this room, to follow along on the webpage, because I'm going to be blasting through like a lot of code. And it doesn't look good on mobile. So I can't get Wi-Fi, but about... about two years ago I was tasked to put Backbone and D3 together for work. And when I started looking into how to do this, there wasn't that many resources online. And maybe I wasn't looking at the right places, but after I kind of went through the first process of trying to use them together, I decided to write a blog post and subsequently, about maybe a year and a half ago, I gave a talk at the Bay Area D3 User Group Meetup about exactly the topic. And so someone came up to me after and said something that I thought was like the best thing ever. And he said after my like 20 minute talk, you know, maybe I'm missing something, but it really doesn't seem that difficult. And I was like, that's awesome. I totally agree. I think so too. I don't think it's anything difficult to use any two things together or any like set of things together. What I do think is challenging and very, very interesting is to figure out how to use two things together well. And in the case of Backbone and D3, I think there's two reasons for that. The first being the obvious fact that Backbone, because it's an NBC framework, is object-oriented, whereas D3 is functional. And second, the point that I'm going to be concentrating on today, is the fact that both these libraries are very, very opinionated about how the data should be fetched, managed, and rendered and updated. So when I was trying to figure out how to show you this in an example, I was like, I don't really want to do the like, you know, to-do app that every, like every website has these days. And I was thinking about, you know, what do I want to do? So my friends and I like to throw parties once in a while, like house parties. And then I was thinking to myself, what if I can obsessively, crazily kind of just track every person that enters and exits our parties throughout the span of the whole party. Now I've never actually done this because the overhead cost is quite high. And I don't think this is going to be fun if you're inebriated, or actually it might be more fun, but we've never actually tried it. I just thought of it for this talk. But because of that, I've prepared some fake data for you. It's an array of objects where each object is in action with the time that the action was taken, the name of the partier who's taking the action, and the action is self. And so the action could either be an enter, so they've entered the party or exited the party. And I made a very simple, very real app visualizing the ebb and flow of our very fake party. So you can see these are buttons from 6 p.m. to midnight, because apparently we party from 6 to midnight, which is not true. But it shows that at 6 p.m., I was at the party with Clarice and Catherine. And then at 7 p.m., our friend Alex arrives. And then so on, and each blue label represents somebody that has entered. A gray label is somebody that's already been there, or had been there, here. And then a yellow label represents somebody that's going to exit to leave in the next hour. So apparently Shirley and I, or me, Alex and I, leave parties really early because we've left at 11. And then at midnight, most people are gone. So here's the code of how I did that with Backbone. It's pretty standard. I created a Backbone collection called Party Collection and gave it the URL that this data, party.json, lives in. And then I've created some very simple functions that crunches the data. So get all times that gives me back all the time values. And get party years at time that if I pass in the time, then it gives me back a list of an array of all the party years, as well as the attributes of if they've entered or if they're going to exit. And then the view itself is also very simple. So I've created a party view where on initialization, I'm passed in the collection, the party collection, and it updates. So there's two ways that it updates. It either updates when the collection sends a reset event, or when a user has clicked on that time button, and we update the selected time, and we recalculate and re-render. And the rendering itself is quite simple also. Get the times and party years, get the data from the collection, and then we just render the time and render the parties. They're very standard. So if we do the same exact thing in D3, so if you plug this code in, it's going to render and do exactly the same things. If we want to do the same things in D3, it looks quite different. So first off, to get the data, we use D3.json to fetch it, and it returns us a response of all of the data as a JavaScript object. Same functions to crunch the data. But when we want to render an update, that's exactly where it looks different. So D3, the way that D3 thinks about rendering is that it comes from the perspective of what data do we have. So we first do a thing called D3.select, or D3.select all. We pass in the selector of the things that we want to grab, and I'm not going to get into too much of the detail of how actually D3 works. So that wraps the set of DOM elements in a D3 selection, and we can then pass it the data that we want it to render. So in this case, the data of the array of times and array of partiers present at that time. And what the .data here does is this thing called data binding, where it has three primary jobs. Number one, calculate all of the DOM elements that need to be entered because it does not currently exist in the DOM, but exists in the data. And number two, calculate everything that should be removed from the DOM selection because it currently exists in the DOM, but no longer exists in the data array. So it is now extraneous. And number three, to bind the data in the data array, the first element in the data array with the first element in the DOM selection, and so on and so forth. So when we render what we want to do is we want to select all the party times, bind the data, and tell it to enter and append the divs that make up the party times. And this creates all of the developments, all the DOM elements that shows all the times and partiers. And updating then is just taking the enter, so we call this the concept of enter, update, exit. So it adds all of the elements that are new in the data, removes all of the elements that are, that no longer exist in the new data, and then update because all of the data, or all of the DOM selections that currently exist in the DOM and also exist in the data will remain in the DOM and only the attributes will be updated. Like maybe the class gets updated or the color. So previously we saw that the color got updated of some of those elements. So hopefully that was a very fast introduction to D3, but hopefully it was good. So before I go on, I want to give you some examples of D3. So D3 stands for data-driven documents, and it's an amazing library that helps you draw visualizations of data. So the first example I have is the D3 demo reel, and it basically takes the data of the past 10 years of closing stock prices, and then kind of transitions it along some of the, I guess, most popular charts that D3 can help you draw. And you also see like the beautiful animations and transitions between each of these graphs. So other than rendering and updating, D3 also provides a lot of sublibraries that help draw, say, axes that make it very, very easy to draw, say, axes on a chart, or to give you scale or make it very easy to draw maps. But one of the favorite things that I like about D3 is the layout library, that if you just pass in data, it calculates the hex and y positions that the element should be placed upon on the screen, and all you have to do is just render them and position them. Like, you don't have to worry about anything. So one of my favorite of those layouts is called the force directed graph. And it just basically, if you pass it an array of nodes and links, it calculates the, it uses this kind of force based simulation between each of the nodes and to calculate its position. And it goes through about a couple thousand iterations, you can configure it to arrive at what it thinks might be an optimal positioning of each of the nodes. So this is why I can't refresh it because I don't have internet, but if you refresh it, you'll see that it kind of like scatters around really crazily before it settles into this optimal state. And that's because it's calculating its positions across all of the different, or across all of the ticks. And this one is just a visualization of all the characters from Les Mis and each link is like a co-occurrence of each character in, or the number of times they, each character co-occur with each other in a chapter. So the thing I like really a lot about this visualization is that it very easily signifies to you that the most important character in the middle has a lot of influence. So this is Valjean in the middle as opposed to somebody on the side here if you drag them around like you can see that they barely make a dent on the, like on the force graph itself. So I have a couple more very beautiful examples. I don't think I have enough time to go through each of them. So if you have the time or if you're interested, please take a look. They're absolutely amazing. And one of the things that I like to drill in about D3 is that it's not a charting library. It's not going to be easy to just like use it and draw like a chart. Like you can't just pass in data and draw a chart or a graph or something. But what it is is it's a set of tools that make the rendering of a date, like rendering of a data set a lot easier, which I think is beautiful because it gives you a lot, a lot of potential to do really creative things. For example, like I'm just going to go really briefly into it. This one over here that you see, one of my favorite visualizations. I've never watched Game of Thrones, never read it. But basically if you look at this, each of the nodes, it represents a character in Game of Thrones. Its size is how many times it's mentioned in that particular chapter. Its shading is from light to dark how dead they are. And it's beautiful. And if you can see these little links, that's a kill. That represents that somebody has killed somebody in that chapter. So a red is a fresh kill and a gray is a past kill. And then you can transition. So then it takes you through like an animation throughout all the different books and chapters that he had access to. And it animates it out for you. Absolutely gorgeous. Please check it out. So after that rant, I want to come back and say what I was previously saying about Backbone and D3. They try to do very similar things but with very different approaches. And I think it comes down to the fact that they have very different motivations. With Backbone, the motivation from what I understand is to make writing a complex web application as simple and elegant as possible. So things like having user input affect the data that's, or even in fact encouraging the fact that user input affects the data being displayed on the page. Being able to persist that data across different sessions. Whereas D3 is all about rendering data, visualizing data, making it easy to do so and making it easy to transition between different states of that data. And it's not as concerned about giving you a framework for if a user input does affect the data that's being displayed, it doesn't give you anything for that. So to me, using Backbone and D3 together is all about if I want to be writing a complex web application where user input can greatly change the data being displayed and visualized. So my first attempt was the attempt about two years ago where I didn't know where to begin so I was like, let's just mash them together. Let's just put D3 right into Backbone and see what happens. And to show you, oh I forgot to introduce this earlier. So I very excitedly did this thing where I call it concise mode. Because you can see that the code is really long. It's about like 250 lines. So I go, concise mode. And then just the very important parts up here. So I'll go through that in a bit. I was just really excited. But here's an update to my previous very simple app example. So you can see I'm no longer just loading in some data. I've expanded it so that you can create the data yourself, reset it, et cetera. And then another thing on the visualization side, the concept of adding another action. So no longer is it just entering and exiting. But perhaps one part a year could be talking to another part a year and we draw a link between them. So maybe, maybe I enter at six and then somebody else enters at that time. And we're like, and Clarice is like, you know, I'm going to talk to Shirley because there's nobody else here. Nice submit. And then there's absolutely no validation code in this. So if you play with it, please be careful because you can very easily break it by like entering people that don't exist or something. So maybe then they're like, oh, I want to talk to Shirley too. Yeah. I'm popular in this world. Sorry. I do what I can. So the code itself remains very similar. Except now I'm fetching from local host. I also have functions to help save and reset. And the only other change is that instead of getting the partiers at a certain time, I get the graph at that, the graph at the time. So I calculate the nodes and links and pass it back. I don't know why I called it app view starting from here by totally mean party view. But that also that code itself looks very similar. And the only thing that has changed is I think event handlers for if a user has input something or submitted something or reset something. So the only big change is that I've now created a separate view called graph view. And I passed it in the SVG element whose class name is graph and also passed it in the same party collection. And the graph view is self the same. So this is the D3 force layout that I'm creating and passing in some configuration. And then same thing just on on change on reset or add or any of those events on the collection just re render for update. And then majority of the you can like it literally is just that I took the D3 code over here and jammed it into say the backbone render the backbone views render function. And so this first attempt it went by pretty well actually it may sense to me as a first attempt. And other than two huge roadblocks I had which will just leave me down to a huge rent. So I've written down like a very detailed blog post. This is the blog post I mentioned earlier about all the roadblocks I went like I got into. And you can read about it in here and I won't say anything because because it would be a long rant. Other than that it worked really really well. Oh like after I figured out those roadblocks it worked really well. So right after I wrote about that blog post the Bay Area D3 user group then like the next day they had meet up on how to use D3 together with other or with MVC frameworks. And I went and I was like where were you about four months ago. But one of the key things that I came out with it is the concept of a reusable chart. So if you've noticed in my first example of D3 code you might have noticed that there's a lot of repeat code. Especially with the fact that when we render when we enter something and then when we update and use enter update exit we pretty much have like 80 percent of the code written and it kind of just gets reused. So Mike Bostock the creator of D3 introduced this concept called reusable chart where basically we just wrap whatever code that needs to be repeated that will be called many times into a function. And then pass it in some configurations. And now we can just use it anywhere from any different places any number of times and you just need to call it and it's pretty like. Yeah. OK. Thank you. But it was a really nice and good and inspirational realization. And so the same example and concise mode. So the collection stays the collection code stays very much the same from the first example. But the but the biggest difference comes in the fact that now instead of a graph view I'm creating a reusable chart that I've called graph visualization and I'm passing in some configurations like within height. And then I'm initializing the graph visualization with the D3 selection for the SVG. So this this dot graph this line grabs the SVG element and I'm just passing it into when I'm initializing it. So then I'm still updating whenever the collection has has updated or triggered and like a reset or add event. But this time around what I'm doing instead of just re-rendering everything I'm crunching the data. And I just say to the graph visualization chart. Hey here's a new updated data now update. And on the other hand with the graph visualization we first have just like a set of the default data. And then here's here's the code of what I've done in initialization. So I'm just setting the configurations on the force layout etc. And then the interesting thing happens in this update code where you know I call update node and I call update link and it goes through the enter update exit pattern and then and then updates the chart. So the thing that I really loved about this concept was now my backbone code and my D3 code was decoupled. So if say tomorrow I decided I want out with D3 or tomorrow I say I want out with backbone. I can just switch any of those like two things in or out very easily without affecting the other too much. And the other thing I think oh actually that's that's a little bit later. So this is where I think I spent a lot of time exploring with side projects of how I wanted to do these things together to use these things together. So the first example I had was last year. There was a Bart strike the subway strike in the Bay Area in last summer and last fall and the same meetup group the D3 meetup group decided to host the hackathon. And the thing that I did with it is this thing where it takes like three different elements and displays it on this chart. And then anything that the user clicks will change the data that's displayed in that chart. So so I used the backbone model religiously trigger change events anytime like religiously backbone view and also this was for example the visualization the code for the visualization for that chart. And it's really ugly like it's really ugly. It's about three hundred sixty four lines of ugliness. So then I kept on being like there's you know there's ways to make this better. So then and please don't think I'm just working on this all day this side projects that I've been doing for fun. And then eventually I arrived at this. This was I think a couple of months ago where I was interested in this is GitHub data so you can input any GitHub user it pulls their top repos and the contributors of those top repos and their repos. And then it looks at their commit history and kind of maps the relation like maps the timeline and relationships between them. So this actually this one this example was very light on backbone because all of the data crunching was happening on the back end. And I didn't really need much of a like backbone model to support that. And so I was using backbone as kind of like a translation layer so translating the data that was coming back into something that D3 could use and visualize. And so the way that I started thinking about D3 and backbone together at this point was that I'm using I'm using backbone for the model for fetching and managing the data. I'm using a backbone view kind of as just the glue between the D3 reusable chart and the model where it just translates things back and forth. And that worked really really well. And I loved it. And until I started working at I started working at this company called Lumio and we do cloud security. It's really cool if you want to check it out. But when I first started there I was tasked to help out with a project called illumination where we wanted to visualize the traffic. So you can't see much but so this is a screenshot of dummy data from illumination where each of these little nodes represents a workload. Each of these links is traffic between these workloads and they're grouped together into these bubbles by either the metadata on them so labels the applications that they're in or environment that they're in etc. Or by the traffic that they share between them. And if a user agrees with the grouping they'll actually they'll create what we call an app container persisted to the back end and then they can operate on them and create their security policies. And meanwhile we've also like we also provided them with a lot of filters to kind of change the view and figure out what they like figure out what they want to see in the environment. But as you can imagine because of all of these things it was the view was just updating consistently there were there were things coming in coming out there were things you can drag things into and out of bubbles and they will change the labels. And there started to be a lot of bugs so we implemented this in the approach number two that I just outlined and there started to be a lot of bugs. And we started to realize that it's because D3 assumes something very important which is that when you update between different sets of data it assumes that the data itself don't necessarily have to live in the same place in memory. And so the way that it calculates the enter and the exit so it calculates those is by either the index of the data or a key that you specify. So in this case for these bubbles we specify that the key was its name so it's kind of we concatenated all the labels together and called it the name. But this started to be problematic because people can rename the name like people can change the different labels and change what the name is so so then the keys were changing and that was causing a lot of bugs for us. So at that point probably most sane and far smarter people would have probably just said OK we'll just give a unique identifier for each of those bubbles and you know and then and then go from there and then and then everything will be fine. But I was like I've been thinking about something for a while and I kind of want to try and see if it works out. So let's give it let's give it a try. And it was the thought process that when D3 calculates enter and exit. So when D3 calculates those datas that's actually very similar to if I had taken an array of data and then told a backbone collection to say set. So we do backbone or we do collection dot set new data and that set method actually returns us and triggers an event for all of the models that have been added all of the models that have been removed. And all of the models that have been merged together and that and thus the attributes have changed. So that's exactly like the enter update exit pattern. And I was like well maybe we should take advantage of that because for our specific example we had back home models that you know weren't changing at all. They were the same throughout all the different updates. And we just want to be visualizing that throughout all the different updates. It seemed like a really good idea in theory. But in execution it was just horrible. To the point that I was kind of debating me like whether or not I should show you because it was just that horrible and that ugly. But then I was like well maybe somebody will be able to see and be like surely you totally did it wrong you could have just done it this way and it would have been so much better. Or validate the fact that it was horrible and I should never do it again. So concise mood. So party collection state the same. But now I've created a graph collection that listens to any changes on the party collection and calculates the graph and calculates the nodes and the links on the graph and then sets them every time that there is a change. So this will trigger the add remove and change events. On the app view side same graph visualization chart. But now I'm not only listening to change events on the collection the party collection. Now I'm listening to add events on the graph graph collection and then and then calling add to graph on the app view or on remove events and calling remove from graph. So this is still within the backbone model I mean backbone view and add to graph just says go through all the models that have been added and call add node on the D3 chart. And then for each of those models also also listen to change events and update node on them. And same thing on remove we just go through all of them call remove node on the reasonable chart. And then on graph visualization the code is still we've just broken them apart we've broken the enter and update and exit code apart from each other but they're still vastly the same. And this works fine too like this work this works fine. But the thing that made it so ugly was the fact that there was just so many now there were so many events being fired from all of these different places that just kind of became a very very ugly ball of mess. So it's now noon and give me about five more minutes and then we can go have lunch. But so when we started realizing this problem it was so bad that our new grad joined and she was like hey Shirley I have this bug and it's because of this event. But I have no idea where this event is coming from. And I was like I'm so sorry I should have just tried just updating those unique IDs but I'm so sorry. And eventually what we decided to do was back in August when I was applying for this for this talk we were using backbone with D3 together. And but we were starting to talk about how we can refactor the code and what we should be using. And we were thinking hey should we stay with backbone or should we consider this new thing that people keep talking about called react you know some little like company made it. Maybe we should try that. And I have to say that we moved to react which is kind of embarrassing to well it's not that embarrassing to me. I've we've been loving it and it actually solves two very important problems for us. The first one being the problem I just mentioned the very tangled mess of of events that were being triggered left and right by that approach. Number three that I took because we use their there they have this idea called flux and we use their kind of a single flow of data. And it's been great for us because we can track where each event is coming from. And number two also important to us is the fact that with D3 we manage the entering and exiting and we have to make sure that that's being done right. And it's great because it gives us a lot of control. But as our app gets more and more complex it just it it's hard to mentally keep track of everything that is going on. And we with react they do exactly the same the same thing calculating entering what's what should be entered and exited and updated with their virtual DOM. And they unit test the shit out of it so we don't have to. And it just seemed like such a no brainer because John Paul says something yesterday about if somebody has built something great why not just leverage off of them and make our lives easier. So that's the end of my talk where I go from backbone and tell you I want to react. But it's been a very very interesting great ride. It's been like an amazing community to be a part of. And I just want to thank you very much for having me here and very much for actually listening to what I have to say. And if you have any comments or you know like you have a bug here or you did something stupid there please let me know either by Twitter or over lunch or something. And hopefully I can talk to you over lunch. Thank you very much.