 Nick Simmons, on here I'm, today I'm here to talk to you about how at Shopify we built a single-page web app and why in the end we decided to go back to a non-single-page web app. So before I start I'd like to say this is actually my first time speaking at a conference so it's very exciting for me. So it's very exciting and also very terrifying so if I'm making any mistakes please bear with me. So let's get started. I'll tell you a little bit more about myself. So first of all I'm Canadian. There's some very stereotypical Canadian things you might have heard of and I would say that I'm very stereotypical Canadian so first of all my favorite sport is hockey. How many people here are hockey fans? Wow five more people than I expected. That's amazing. Second of all Canada's not really that well known for food but maple syrup my absolutely favorite thing to eat. Whenever I have pancakes or waffles if I don't have maple syrup I'm extremely disappointed. And last of all in Canada we unfortunately have to deal with winter. So this is a picture of the city where I live and you see those things there on the right those are actually cars. So a very important thing when you live in Canada is to get underground parking. Cannot tell you how important that is. So other interesting fact about me is that I actually used to live in Singapore. So I was here for about a year and a half and I was working for LucasArts so it was super cool getting to work on Star Wars stuff living on the other side of the planet getting to travel a lot. It was great. I was actually a back-end developer for some online games that we were working on. Unfortunately none of them shipped and there's a good reason for that. About six months after I had started at LucasArts we were acquired by Disney. So you know Disney. Disney seems like a really nice you know family-oriented company and they are but they're a little bit ruthless when it comes to business. So when they bought LucasArts or Lucasfilm I should say they decided that they were really into the movie idea but not so much into making games. So in the end well that was the end of that. But it's on all bad news. In the end I ended up moving back to Canada and got this awesome gig at Shopify. So this is basically what I'm doing now. I've been at Shopify for a little over a year and this is actually the first time I started working with Ruby in Rails. So I'm still relatively new to it. Actually another interesting fact is that it was also the first time I've ever used a Mac. Not programming Mac. I mean literally using a Mac. I had no idea how to even install an application. But yeah it's been a wild ride and I really loved it. So specifically at Shopify I'm working on the admin interface. So this gives you an idea what it looks like. Basically it's a pretty large web app. There's lots of different sections. Lots of different configurations that our shop owners can do. So as I said, lots of pages basically all form submissions and lots and lots of data going on. So I'd like to talk a little bit about the evolution of the Shopify admin. So in the early days the classic version of admin was Rails app using some templating with ERB and some jQuery sprinkled around to do any sort of cool stuff we wanted to do in the client. But a couple years back a decision was made that they wanted to build a single page web app. Now the original reasoning for that was they were thinking well hey if we could have the same application for both mobile devices and for the web that would be great. Basically just to abstract the client away from the server and also to be able to build our JSON API which at the time did not exist and just treat the client as another consumer of this API. So when they were initially evaluating the JS NBC frameworks that existed, many of the solutions that were out there at the time really met the needs for the team. So they decided to build their own. And from this Batman JS was created. So when we started building admin 2, this was still before my time, Batman JS was the framework that it was built on top of. So we finally released admin 2 in July of last year, 2013. And funny enough six months after releasing admin 2 we decided that it was time to move away from this framework. So our new admin is called admin next because I'm getting tired of using numbers. So you might be curious what admin next is about. Well I'll talk about that soon. But when I say soon I'll only briefly mention it in this talk at the end. To be honest we're not completely finished with what we're doing. And I'd like to make sure that in the future we'll give another talk. We'll explain exactly what we've done and hopefully we'll open source it. So start my story with when I began working at Shopify. So when I come in I was sort of interested in the idea of doing a JS and VC app and a single page web app. It sounded like a really cool thing. I was really interested in it. But I never actually got to work on anything like that before. So when I joined Shopify, Batman was basically finished. Admin 2 was almost out the door. And everyone was really excited about this new direction that we were going. I mean with this we had no more full page reloads on pain transitions. That's really cool. All of our HTML endpoints that we had in the code were completely removed. And we had all of our templates stored in the client. JSON API which was shared between admin as well as any third party apps. So that was cool. Basically the server no longer cares about any of the UI concerns and the admin API. I said it's just another API client. Great. But there's a little problem. Batman is super hard to use. At first I thought it was just me. Okay, I'm new to this stuff. I've never done it before. So just there's a learning curve there. I'll pick it up eventually. But it turned out that a lot of people were having this problem. So when I first started at Shopify, I assume there would be a certain learning curve and it would look something like this. So over time, I would slowly pick up more and more of the framework and have an understanding of what works and by the end of it maybe a couple months later, I feel like I'm basically an expert. Unfortunately, I found my learning was a little bit like this. So it took a lot longer to wrap my head around what was going on. So to explain this sort of learning curve, initially, as most of us do, whenever you need to solve a problem, you try to look for other examples throughout the code base. The problem was admin 2 is a pretty large code base. So there's a lot of stuff going on because of the complexity of Batman stuff living in multiple places. It was really hard to sort of piece everything together and figure out what's going on. So the next logical step is, all right, let's take into the into the docs. So Batman docs existed. But I would say they were far from complete. There were many pieces that were missing. And the most important part was that there wasn't really a good onboarding into Batman, you couldn't just follow a simple tutorial, get started and have a Batman app up and running very quickly. So I gathered a little bit from that, good overviews, some of the some of the things were covered. So I felt, okay, I don't have a full understanding. But I think I'm ready to dig back into the admin code. And shortly after doing that, I started running into those things are undocumented. So now I'm kind of stuck. What do I do? Alright, well, I'm on the admin team, obviously, there's people on the team that can help me out. So I started asking for help. Problem was, is that Batman was built by several developers at Shopify, and most of them had actually moved on to other projects. So on the admin team, there were two people that were really the experts at Batman, and the rest of them were all new hires, including me. So there just wasn't enough experts to go around to help us out all the time. So finally, the ultimate way to learn how to use Batman was to dig into the source code of Batman itself. Now, in the end, this is always a good idea if you really want to understand the underlying stuff you're doing. But as a newbie trying to ramp up and be productive quickly, this was a daunting challenge and a very slow. It took a long time to get through it. So a month later was hack days. So hack days, at Shopify, you get a couple days of one of the company can work on whatever they want to. And a lot of people had decided that with I've been to shipping soon, that they wanted to try out Batman for their own projects and see what they could do with it. So people were like, yeah, let's use Batman. Let's see what we can do. So okay, where do I start? They followed the same natural progression as me. Okay, let's look in the docs, you know, let's get something up running. Okay, that's not really working so well. So oh, crap, someone please help me please help. And of course, a lot of them were coming to me. And this was still at the very beginning of my learning curve, I had no idea what was going on, still figuring stuff out. So it turned out that most people were just like, well, that's enough of that. And nobody ended up using Batman for the project. So that was definitely like a big hit to the team. Like, wow, that sucks, you know, this is what we're moving forward with. And other people in the company aren't really comfortable using it. So the first big lesson we learned was that solid documentation and excellent examples are so important if you want people to use your framework. Brandon the other day had touched on this, really to guarding your open source projects. Well, we were not very good gardeners. So after hack days, we set about improving our documentation, making sure that all the gaps were filled in. And we moved on from there. So for the next couple of months, after we shipped admin to the focus was on new features, we had to expand our feature set and keep our shop owners happy. Now, as I mentioned before, most of the admin team was actually new to Shopify. And on top of that, many of them were new to Rails. So we were spending most of our time in the client, which is written in CoffeeScript. The thing with Batman is Batman, our approach or the approach that they take with Batman was that it was MVC to the max. So we had all of our models, including all the associations in the client. You had the controllers and you had view state that everything you could possibly ask for. So that meant that we could do a lot of things in the client. So first question we'd always ask ourself on a new feature is, where should we put this code? Well, in front of Dev, the client has all the state that I need. So let's do what we can there. Now, there's an obvious problem that comes up with this. We weren't just building our features for the admin. A lot of these features would be used by other applications. So along comes POS, which is our point of sale application. They were developing in parallel to the admin, maybe a couple months behind us. So what happened is they will come along to a feature that they needed to implement that we'd already done in the web admin and say, oh, okay, well, not everything is supported in the API. So we're kind of comfortable developing in Objective C. So they followed the same path as us and implemented stuff in their client. A lot of duplication, not ideal, but the worst part was that there were just these tiny subtle differences between how we implemented it and how they implemented it, which ended up leading to a lot of bugs, specifically related to how we handled orders. So we have problems like this. It becomes very difficult for third-party, especially third-party apps, that are pulling down this data and then trying to understand what are the subtle discrepancies and why is this data not all look the same. So we weren't doing a very good job of enforcing consistency throughout our system. So in the end, what we really should have done is we should have been moving as much of this business logic into the server. The server is the authority on the data and on top of that you can't really trust the client anyways. The client, especially in a web client, all of your code is just there and someone can go in and manipulate it and do whatever they want. So the second big lesson that we learned is that we really need to minimize the business logic we have in the client. Put stuff into the API. So it was around December 2013 that I personally started my super deep dive into Batman towards the end of my learning curve in the graph I had before and trying to really understand the fundamentals of how it works and what I can do to help make it better. At the same time, there were a lot of people on the team that were sending to say, you know what, the admin looks pretty janky. It's not loading as fast or smooth as we would have hoped. So, you know, what's going on here. The expectation was that the initial load, downloading all the assets, would be a little bit slow. But from that on, it should be pretty smooth. So here's where I have a little video that shows the first clip shows what we would hope that the admin would look like. And the second part of it shows what was actually happening in admin 2. So here I'm clicking on a link going to an order. When the order loads, everything snapped into place. Now here, this is in admin 2. You noticed that some of the sections sort of popped in over time. It wasn't all just like loading immediately. So let me show that one more time. So this is the desired effect we would hope to have. Everything's just there. And this is what was happening in admin 2. So we considered this a form of jank. It was not something that we wanted to happen. And actually, it's a little bit of foreshadowing. The first example was how admin next is working. But the second one was admin 2. So the reason behind this was related to how Batman's binding and rendering was working. So for any of you that aren't familiar with bindings, I have another quick example here that shows a great application of a binding system. So here, I'm typing in something in the title. You'll notice down below it's a little hard to see that that section was being updated at the same time. So we have some client state. Hasn't been persistent to server yet. We want to make sure that those two fields are tied together. That is a great example of how you can use bindings to basically tie two fields together or possibly to link some of your JavaScript objects to HTML. So now if we look at another example of the order page in the admin, how many bindings do we have on this page? Well, let's see. We've got a binding up here for the time stamp of when this order was created. We also have another binding here for whether it was a test order or not. We just made a little warning message. Some more bindings here related to the line items on the order and the details of the pricing, for tags, notes that we can associate, the address, and also all of these events. So there's a hell of a lot of things going on there. But how many of those bindings are actually like dynamic and just kind of happening on the client? Well, the reality is all of that stuff only changes if we do a request to the server and when we get back the response, then we apply any updates on the bindings. So the thing is like, wow, we kind of do the same thing if we just done an HTML request and let the server take care of it. So not really getting much on this page out of the binding system. And the problem with that as well is that there's an overhead to having this sort of binding system in the client and rendering everything in the client. One of the primary overheads that we had was our asset bundle. So you might be wondering, well, how do you, how are you rendering all of this stuff client side? Where's all your HTML coming from? Our solution was to bundle every single page in our app, which as you saw was a lot from the first slide I showed how the admin looked. Every single page, whether you use it or not, was bundled together, converted into basically a string and put into a JS file and bundled up like that. So we had, our two main ones was Batman itself and the admin two templates. So this is a lot of overhead for possibly a user that's only visiting maybe one or two pages. Now obviously you have some advantages if you, if you, some caching in the browser, but at Shopify we tend to deploy a lot, sometimes a dozen times a day. So the cache is constantly being busted as we change these templates and for someone that's using a very slow client or a slow connection 3D or just doesn't have very good internet, this is obviously a lot of overhead that's not really desirable. So the big lesson we learned in the end was that certain to feel like JS and VC, we still like the idea of it but wasn't really working that well for our app. We had a lot of state that was only changing when we do a quest response from the server and there was all this overhead without really that much of a benefit. So everyone agreed that there was some parts of it that were really great and a big improvement over admin classic. The main one being bindings. There were certain cases where we really wanted to keep those bindings. The first example I had shown with basically updating a title and another section is being updated, that was a great case. This is not something that's coming from the server, something that's changing dynamically in the browser, that's great. But for using bindings to update stuff when it comes back on the server, well it wasn't really helping us that much. So in the end we decided that pros versus cons, it was better off to see if we could try a new approach, try to get all the same benefits that we had in MN2 but at a much cheaper cost in terms of asset size and also complexity. So a decision was made to reboot the admin and kill Batman. So some of the key goals for admin next was it had to be easy to use. We learned from that hack days that people really struggled ramping up and onboarding with Batman. So any new framework we would build, it had to be something that we could bring in someone new, they could mess around with it for a day and be comfortable using it. We wanted to keep the good bits, so full page reloads, didn't like that. We definitely wanted to keep the ability to have pages, basically just load the sections that we needed to change, but at the same time minimize our client states. Really wanted to make sure that if we were going to make the switch, it better at least be as fast as Batman or better. So the other key thing what we were thinking is that at Shopify we become very, very good at scaling the server. So by building Batman and pushing more stuff to the client, one of the big challenges we had is that we have no control over what our customers are using. So they might be using a super fast Mac, MacBook Pro, or they might be using some crappy phone that's only supporting 3G. So because we have no control over that, there's only a limited amount we can do to help scale the client, but on the server, we can just keep throwing more servers at it or we can improve caching or whatever, so pushing the rendering back to the server seemed like a good idea. So what exactly has that been next? Well, for the most part, it's just Rails. Plus, we started using TurboLinks. This is the part where I'm expecting somebody to throw something at me. But to be fair, we took TurboLinks, the idea of TurboLinks, and modified it for our own purposes. So the general idea of TurboLinks is basically that instead of doing a full page reload, it just does an AJAX request and replaces the body. So we like that idea, but that was a little bit too simple. We wanted the ability to replace specific sections of the admin, depending on what was happening. So a good example of that is, let's say we update an order, update the let's say we capture some funds. The only thing that really changes is the total amount that's been captured, maybe a small little text section and something simple like that. So we added the ability, similar to how PJAX works actually, that we could just specifically replace a set of nodes on the page with the response that comes back from the server. But on top of that, we weren't forced on the server side to necessarily understand what the client was doing. So the server could render any amount of the HTML, respond back with it, and the client would be able to decide what parts of that HTML it wanted to insert into the DOM. So we're still using the same JSON API endpoints that we had before, just the difference now is that if you're in the admin, it's responding back with HTML. And as I mentioned, we wanted to keep a binding system. So we built a new binding system in only 255 lines of JavaScript, very, very simple and basically works the same as it did in Batman. So that's what we've called bindings.coffee. So really what we have here is not necessarily a full-size framework, we're just using a few simple technologies to achieve the same goal as what we did with Batman. So the big lesson we learned in doing this reboot is we really want to try to keep things as simple as possible so that it scales well. Now when I say scale, I don't just mean the performance of in the browser. I also mean scaling with the number of employees we have at Shopify. We're growing rapidly and it's really important that we can onboard new people quickly and have them comfortable working in our stack. So how's it going so far? Well, we're not finished yet. We've got a couple of sections that we've shipped. We've seen great results so far. The biggest difference that we've seen is that people are saying, wow, this is just so easy to use. And that's great after our experiences with Batman. The client side logic has been significantly reduced. Most of the decisions we make now, we can just do an ERB. Now, any logic that we do have client side is specifically related to UI concerns. And even after building some of our most complicated views, specifically being orders, we found that very, very little client side code has been added. So the fact that the client is just so simple to understand and so little code and we still have all the same functionality with Batman is really a testament to how, sorry, lost my train of thought. Anyways, it's been a great thing. So before we shipped this open source, we definitely want to approve the documentation a lot more. We don't want to make the same mistake we did with Batman. There's certainly some optimizations we can do. As I mentioned, when we're rendering stuff from the server, sometimes we're rendering far more HTML than is actually required by the client. So there's some things we could do there. But overall, we're very satisfied with our decision to move away from a full single page web app. Now, I'm saying it didn't really work for us, but I still believe that JSNBC makes a lot of sense for certain applications. In our case, it wasn't for us. So Batman is dead, but for us, it's terrible time. Thank you. Thanks, Nick. Any questions for him? A question for you. Would you reconsider doing a single page app with the developments in a lot of frameworks such as Ember, RequireJS, Angular, and stuff like that? For me personally, yes. I would be interested in exploring other frameworks. I think part of it now is that with our experience with Batman, we've sort of shied away from it. But I do believe that as JSNBC frameworks are improving, that a lot of these problems are being solved that we had. I would actually attribute a lot of our problems specifically to Batman and how we've built Batman, not necessarily JSNBC frameworks as a whole. Cool. A second question if it doesn't look like anyone else. With how much of a role would you say Jason API play in the decisions he made on the front end and stuff like that as far as how your data reloads and stuff like that? Sorry, in terms of what? The way in which your data is structured and what you send through to your front end. Say if you're sending IDs that then require another fetch from the API for more data. Like you showed that page where it was loading data in convention. Was that just an artifact of the way you designed your API? Actually the API was returning everything in one response. So we just had one JSON request that was coming back. The problem that you were seeing there was totally related to how Batman was handling that response. It's related to some of the details of how we did some of the bindings, how the bindings would basically render render out the HTML. It's beyond the scope of this talk for sure. Some deep complexities in there that I would prefer to forget. But yeah, it wasn't related to how the API was responding. Hey, did you try to contribute to Batman or like contact maintainers and talk to them about these issues that you had? I mean, I'm wondering if it's developed and if your experience somehow made maintainers to change, to improve the library? So actually, it was around December when I was doing that Batman deep dive that we had started to discover some of the flaws in Batman. And I had started to come up with some solutions for this. But there was a lot of work that would have been involved. One of the problems with Batman is the amount of complexity that was built into there was a very large framework. There was a lot of things going on and making any change was very difficult. Whereas we figured it's probably going to be just as expensive to just do a reboot and go from there. So that's why we went the prototyping solution. Any more questions? If not, thank you, Nick.