 Helo! I've been here a few times, lots of people talked about Edinburgh Data. If you Google it you'll find that it's sort of alright, but it doesn't really do what everyone needs, and it's only in beta and you shouldn't really use it, go and roll your own. We've been using it now for six months or so, and I've actually found that once you get under the hood and you know where to go, it's really, really powerful. The real problem with it is it's not documented very well. It has a huge amount of functionality and I'm going to go through some of that functionality tonight and show you how it can be used if your server isn't the clean Ruby loveliness that you built yesterday, but it's actually something that someone else built six years ago, in our case, 15 years ago, and how you can use that with Edinburgh Data. So, I'm from Silvercurve, I'm CTO and CEO, and the idiot that funds it. We're experts in digital signage, so replacing posters with screens. We run the mother care network, I used to run the O2 network through 2000 shops, and we have created a graphics engine that allows you to do on a Raspberry Pi what you used to have to use an i7 with a quad core graphics card to do. And that gives you PC quality graphics at a fraction of the cost, about £750 cheaper over five years per screen, as a guideline is three and a half thousand screens on escalators in the underground. So, they would save close to £2.5 million of five years, most of that in power, by going from a 75-watt device to a 3-watt device. And I'll give you a very quick demo of what our device can do. A standard Raspberry Pi, nothing tricked out on it. Once the projector's caught up. So, this is rendering in real time on Raspberry Pi. There's no video in this demo. We've got strokes on text, we've got the ability to bind text to data. We can do pixel shader ripples in about ten lines of JavaScript. We've got a dead smooth ticker. I challenge anyone to do that in HTML5, even on an i7. What we're using at the back end is a language called QML, which is a declarative JavaScript language, rather than in HTML5 saying, right, move, move, move, move, move. Instead, you just say, I want to go from here to here, over ten seconds on this 3D path with this blur and this fade, go. And the engine gets on and does that on the GPU. And you'll find on these mobile phone chipsets about 95% of the chip is GPU, about 5% CPU. So, anything that's HTML-based is going to really struggle. That's why we exist. If I just switch that over, we have a second demo, which, if I can get to it, really shows off some of the rendering capabilities. It takes a little while to load. In this demo, there's nothing that's video. It's all rendered in real time on an $8 chip, eventually. We're using C++ at the back end for this one, but then our user interfaces are built in HTML5 and JavaScript with Ember running in the user's browser where there's plenty of horsepower to play with. What we're trying to do is offload as much as we can off the device itself onto the user's browser. So, here we go. This is the sort of thing that we can do with our engine. There's a particle engine at the back with a bokeh effect blurring in real time on those PNGs. And then, in-store media is our Swedish software partner. They implement their platform on top of our engine. So, here we've got a piece of text that's editable by the user. It's about 30 milliseconds from editing it in the browser to having it on screen with a ripple as a reflection. So, we're rendering it twice, adding a pixel shader on with the water rippling as well. Hitting 60 frames a second full HD. Complete with beautifully smooth animation. So, you sort of CSS three type animations, but put those on steroids. And one more. So, this is the sort of thing you might see in a restaurant. We're actually about to use this for the bokeh menu board network. What we allow basically is the graphic designer to get at this in a really, really easy to use way without having to write any code. And then the programmer can go and build new effects for the graphic designers to use. So, that's our main business. To achieve that, we need to be able to allow the user to control it. So, we use Ember in two places. We've got a third party piece of software that's licensing our engine. And their player is running on the Raspberry Pi. Under their player is our aperture engine rendering pixels out to the screen. And that third party server might be in the data centre for the client. So, Mark Spenser run it in their own data centre. Mothercare run it in the cloud that we host for them. And this particular one is a Swedish software company. They have their own HDMI 5 front end, but it is suitable really only for the expert. It took me about six months to learn how to use it. So, we've written our own content management tool on top of their APIs. And we've done that in Ember. We've also got a second interface, which is on the player itself. So, the physical player will sit behind the screen. The engineer, when they install it, has set up the network, set up the screen resolution, detect what the screen is capable of, turn it upside down if they hung the screen upside down, all those sorts of things. So, very much like you have a UI on your router at home, we have a UI on the device. The device itself has some static HTML and JS, which is the Ember application, and then some PHP APIs that drill down into the Linux commands to actually go and change the host name, change the IP configuration, that sort of thing. So, if I show you what that UI looks like, this is the cloud UI. So, this is content management. Let me just zoom in on that. Feel that me? That's exciting not to zoom in today. So, here we have two views. We have a dashboard, which tells me what the players are doing. So, here we've got two players that we've disconnected in the office, one of them sitting here, which is why he's not talking, and then we've got one here that's okay. We can tell the IP addresses, serial numbers, that sort of thing. So, those are talking up to the server every minute or so, telling them what they're doing and asking for new content, and then we're just pulling down Jason about them and displaying it. In the content view, we're showing the slides. So, each one of these is a piece of content on the screen, and the graphic designer can create templates for those slides, those templates have the layout of the text and the images and the animations and the duration, that sort of thing. So, if we take this one, for example, this has got, you'll see top left, a little preview, which I can zoom up so you can see it in more detail. So, we have a title and subtitles, a very simple demo, and two images that can be uploaded by the user. So, the graphic designer has set up the layout of that slide, and then we're going to populate that with content in this user interface. So, first of all, we have a slide name, which is how the user is going to see it. That never appears on screen. If it's draft, then it's not sent to the screen. If it's published, it's going to be seen on screen. Obviously, very important if you're running a retail network, you don't want next month's promotion to go up early. We've got weighting of play out, so we can play it more or less often than other content. And then we have the data fields that are actually editable from the template. So, we've queried the graphic designer's description file, which is a JSON file help in the template, and it'll tell us what properties are available and what types they are. So, there might be some text fields, some image fields, a video field, some numbers, a checkbox, whatever it might be. And we're then representing those in the UI. So, here I can change in a brasary in a cafe because it's stupid to use a French word. And then I can go and choose a file for example. I can also change... Is the schedule gone? Oh, you've moved it. So, we can change the scheduling rules so I can set the dates that it's available to be on air. If your sale finishes, you don't want the sale advert to come up the next day. And then which day is the week and that sort of thing. So, this is some nice little multi-select options. So, there's a lot more complexity to it, but I'll leave it at that. So, to do that, we are querying back to third-party APIs that someone else wrote on a system that's been developed over about the last 18 years. So, we've gone through a lot of technology changes in that time, and some of the APIs date back 18 years, some of them were written last week. So, we need to be able to query all that data, bring it together in the browser, and then render it beautifully. So, EmberData conceptually gives us a briefcase model, i.e. I go home from the office, I've got pile of papers in my briefcase, there's a subset of what's in the office, and I want to be able to edit those documents, put them back in the briefcase, and when I get back to the office, sync them to the storage in the office. That's essentially what you're doing in the browser whenever you bring this data down, mungit, send it back. The difference in EmberData is you can store it for quite a long time in the browser, and then persist them back, and you can keep them after you've persisted them. So, the persistent data means that if I visit a page, show some data, go and do something else, come back, that data is already in the browser. It queried it the first time, and it's kept it around for us. We don't have to re-query it, so we give a much more slick user experience. We did have transactions, which I think are a really useful thing in a briefcase model. If I sign three documents, if I sign the third one, I want to roll back that whole transaction. They've been deprecated in 1.0. I don't know if anyone's aware what the long-term plans are for that. We'll see what happens with those. We've got lazy loading. So, if I've got, for example, a directory tree that wants to go and get hold of, I don't want to load every single directory just in case the user wants to look at it. As they open up a folder, I want to bring back just the stuff that's in that folder or just the subfolders and EmberData gives me all of that out of the box. We can customise it. EmberData has a certain way of doing things. If you don't like it or it's not right for your project, you've got the ability to go and change the way it works, and that's what we've done very heavily in our project. Now, EmberData out of the box likes REST and a very specific format of REST. So, your request to the store, you would find wizard number one, and then when that promise completes, you can get him to Wave his wand. That's going to go to the REST adapter, which is part of EmberData, and that will form a URL, wizard slash one, and ask for a URL get. Now, that obviously assumes that that's the correct URL for your server. Now, if you're writing your own server, just write it the way EmberData likes it. But if you're using something that is an existing API or someone else's server, you don't have that choice. The next thing that happens is that the response comes back and it has to look like that. So, we need a wizard or an array of wizards where one of them is ID1 with a first name and last name, and that will get pumped into a model, again named wizard. We have a first name and last name string attribute on it. So, that first name will get pumped into the first name for you. So, all the code you have to write to use it is that all of that is done by EmberData and your server. So, that's a model as well, but all you have to do is make sure the names in the data coming in match the names in the model. Now, obviously, that's only going to work if you write the server, because no one else writes code exactly like that. So, it's all easy peasy lemon squeezy. So, if you didn't write the server, you need to start doing some work. And that's where you start hitting the myths in EmberData. If you Google EmberData, embedded, polymorphic, lazy load, you'll find 100 posts saying it can't be done, and one little tiny post on page 50 saying it can be done. So, the myth is it can only do rest. You'll find lots of places where it says use the rest adapter, if you're not rest, don't use EmberData, roll your own. It will say everywhere it can't do embedded records, and that's true of EmberData 0.1.2, but now it can. It's not obvious how to do it, but it can do it. And it definitely can't do polymorphism. Absolutely impossible, but if you know where to look, it is entirely possible. So, if we lift the bonnet and have a look underneath, third party APIs are, let's call them challenging. We, in our project, probably about 80% of those that we list on screen, we're hitting in our project. We've got inconsistent signatures. We've got get and set, have a capital letter or a lower cases for the same API. Some of the schemers, we push a blob of JSON. Some of them, we encode that into a string and put it in the URL. We've seen strings with a blob of JSON sitting inside the string. And I wouldn't be surprised if we found a huge blob of XML sitting inside a string inside a JSON object contained in a URL. You'll see, if you Google EmberData, that it has problems with trees of infinite depths. In fact, it doesn't at all. In our case, we have objects that have an ID, but they also have a version. The whole thing is a version-controlled object store. And that gets very complicated because your URLs get cached and the caches have to be adjusted for version. So, that gets a bit of fun. And ours is polymorphic as well. And you saw that in the demo where those properties were listed for the slide. There was a text property, an image property, a video property. The slide just has a list of properties. So, we have polymorphism there straight away. So, for given example, embedded data and strings is probably a bit small, but we've got a value here that's got a JSON object sitting inside it, inside a text string, which means we need to parse it and do all the horrible stuff with it. We've got polymorphic data, and this is the data behind what we just saw in the demo. So, we've got an item list. We've got a text title, which is a type text. We've got a value, but in the case of a file, we've got type image, which has a file ID instead of a value. So, each of our different types of model have to go and parse the data differently. And we've got non-standard URLs. This one's a particularly fun one. Whoever wrote that had had a lot of beer, I think. We notified him that, and he said, yeah, it might cause some bugs parsing that. Yeah. So, on the left hand side, you have your root and your controller doing all your beautiful data manipulation and pumping into your views. And on this side, you're working with models, DS.model descendants. You're talking to the store. Now, the store is your briefcase. So, you open it up and you pull out whatever's inside it. But if there's a document that you want that isn't in the briefcase, you just open it up and pull it out. It's magically there. The find method on the store will take a type and an ID. So, if I want document number seven, I just ask for find document, number seven. And it returns a promise. If it's got it, it'll just immediately resolve the promise and there's my data. If it hasn't got it, it'll go back to this lot and request it and bring it back to me. But it brings it back exactly the same to this side as whether it had it immediately or whether it had to go back to the server and get it. On this side, we deal with records which are plain old JavaScript objects. The basic is a name mapping between the two. And on this side, we don't deal with models at all. We only deal with records. So, the adapter is responsible for communicating with the server. It's building URLs, it's requesting the data and in very simple cases it could munch the data into the format that the embedded data wants. But I would sorry recommend you don't do that. The serializer's job is to munch the data. So, it doesn't talk to the server, it just gets given a blob of data from the adapter in server format and it sends it back in store format. Which is the nice beautiful format that you'll see when you read the embedded data docs. You just bring it back as if it came from a nice clean rest server. When you're saving data it takes the store data, munges it through the serializer into server format and then pushes it back to the adapter. And then we push off the server. And that's true in almost all cases. There are some cases where you might have a single model in the store but it's actually stored in two or three different places in the server. Maybe half of it's stored in one place and half it's stored somewhere else. In that case, the adapter isn't restricted to making one call. You can make as many calls as you like. You can put it later in the serializer into, for example, three child JavaScript objects. And then the adapter can send each one off in three separate calls to the server. And you have to deal with what happens if one of those calls fails. But that's the fun of programming. So Andy here is our lead ember developer. And he's going to talk through some of the examples of how we've used these technologies to build our code. Great. Basically, I'm just going to show you the ways we've dealt with some of these problems. As Brian said, we've got this problem with embedded records, supposedly. You can have the kind of code on the left coming back from the server and by default, ember data doesn't really know what to do with that. It just kind of falls over and you probably get some cryptic error message. It does like what you've got on the right. So what we have is this side loading technique where we've basically stripped out the template data and we've put it into an object alongside our slide. And ember will play with that quite happily. That is something that we have found to be done quite nicely in the serializer. Just because it's manipulation of the response that's come from the server. So just to get a little bit more involved. This kind of relates to what Brian showed you earlier, that list of different properties in our UI. So what we can have is a definition of models as we have in the yellow text up there. So we have a template which quite simply has a list of template properties. And the important thing there is we have we pass in the polymorphic option into our has many and just a side note that little object that you pass in there can be very, very useful. You can basically put arbitrary stuff in there and pull it out later if you need to flag certain properties or what have you. So it's pretty handy. There are a few built-in properties that Ember data makes use of out the box. Then if we have we define just a base template property model and then just in our example we derive a text template from that. Actually no we don't because that's a mistake in the code there. That should be over here should be an app.template property. You get the idea. One person noticed that before we did. Really? Yeah. Who was it? So yes. You can do that. We've got our model set up and that's nice and easy. Then let's say we have the response on the left there from the server when we look up this particular template. Again we can simply just go through and do our extract those embedded properties into side-loaded data there. The key thing is that instead of just a numerical string ID we have an object which has an ID and a type. Although it's not really documented anywhere the combination of this and this flag basically just makes Ember data happy that's all there is to it really. You can quite easily get a polymorphic list there for example. So that's not as bad as people made out. Now with non-standard URLs there are quite a few ways of dealing with it depending on how horrendous the API is. If you have the standard restful ending to your URL then you can just derive from the built-in rest adapter and you can supply namespace and a host for example if you need to. Then when Ember data goes off and builds a URL to make a request to it will take those into account and you'll get the request made to the right URL. If it's a little bit less restful you've got the chance to the option of overriding rest adapters build URL method and really you can return whatever you want in there and then Ember data will make its request as it normally would to the URL that you have built so that allows you quite a high degree of control over your URLs. An alternative if for example something that we've come across with this API that we're working with is we need to make two, three, maybe more different requests to get even seemingly linked bits of data for one model so that doesn't really play that well with Ember data on its own so what we can do if we're finding a particular or if we're calling a find we've got find, find, or find query that kind of method we can basically override the Ajax request that is made by the default rest adapter so we can have our own custom adapter override any of those methods we like and then we can kind of do whatever we want in there and you've got nice things for situations where you have to for example make a number of requests for one particular find query for example things like you've got Ember's RSVP methods something like hash is quite nice there you can take the promises from your three requests put them into one hash that whole thing to resolve so that's quite a nice little trick same goes for create record update record, delete record you can basically do what you like I'll show you an example in a moment of where we've had to override create record because because of this API I'm not going to say anything bad then you've got the issue of non-standard data the easy way of handling it is to basically butcher it ourselves into the form that we wanted him a nice little thing that we came across that made our lives a lot easier was a way of basically stripping the data that we get back from the server and putting it into a really nice restful format that Ember data is happy with so as it says there and that was a big step in simplifying the way that our data is handled we've also got options of overriding methods on the serializer so you can extend serializer by default Ember data will use the rest serializer but you can go from just the base serializer if you're happy implementing a number of methods yourself you have quite a number of possibilities in the serializer for controlling how you handle your data and I'm going to be perfectly honest here I haven't quite sussed out what is the correct way of doing this you have extract single and extract many methods which are called by Ember data in response to either a find or find query for example which will give you it could give you a list of objects back and actually that should be extract array rather than extract many and that's what they'll then get called on your serializer the list of returned objects gets passed in and you can do what you want with it in there same thing happens with extract single if you do find by ID for example you'll only get one object back extract single gets called to process that payload you've also got normalized payload and I think there's a normalized hash method as well which I haven't actually really looked into and they again give you a chance to play around with the objects that you get back from your requests so the purpose of these things as far as we're concerned is to rearrange the data that we get back from the server into a format that Ember data will happily play with without us having to make any other changes down the line so I'm going to show you some real code this is probably actually going to be too small so that's way too small isn't it keep zooming can you how? same way let me just grab should probably have found this earlier look at that that's better so we have this adapter and it goes easy on me it was the first thing I wrote out of uni so it's evolved shall we say from its initial implementation now we've got some examples of issues that we faced inside our create record there's probably a better way of doing this so if anyone knows it please talk to me about it now the first thing that we've done here which is a little bit non-standard is that our serialisation takes place inside our create record method seems a little bit dirty to be perfectly honest there's a reason for it which is that we need to upload files to the server and we need the ID that the server assigns that file before we can persist our actual slide data and obviously uploading a file is not hugely synchronous so we have in this case an asynchronous serialisation method because by default serialisation is synchronous so we can do this we can wait around for it when we're done with that we need to make an Ajax request so we've kind of just built this wrapper for getting a URL because we again have to deal with non-standard URLs so we've this is effectively an override for something like the rest adapters build URL it's just we kind of rolled our own before we really realised that you could just override what was already there so that gives us our URL to make our request once we get that back we unfortunately have to make a second request simply because all of our the data is stored in nested data sets and the first call unfortunately won't allow you to specify parent for the newly created one so we have to create one and then append it to its parent which unfortunately we couldn't find out from the API documentation so there are a lot of orphans floating around in our test space so that covers kind of multiple calls within a single operation non-standard URLs also the nastiness of having asynchronous serialisation we have trying to think if we've got any other examples we've got some serialiser stuff which I don't think is actually particularly interesting it's worth looking at the side loading piece back end objects so we've got this our own little base class which we've derived from rest serialiser now this is basically what we have to go through and pull out embedded records and pop them into side loaded data so we just have a few methods in there which we'll go through and do that fairly straightforward stuff just kind of handle with so handling relationships so for example we specify that we've got an embedded relationship we then check if it's a one to one or if it's a has many for example we'll we'll handle that accordingly this options object you see here is the one you saw earlier with a polymorphic flag in it and look at any flags you like and this is actually iterating the model and looking at the metadata of the model to see what it should do with it this code is heavily based on someone else's code and we couldn't actually find tonight where we got it from but if you google embedded rest adapter about 99% of that is in here we've made some mods for our case yeah I don't think there's a huge amount more to show if I missed anything um the polymorphic embedded models okay so and your Ajax mixing what we have is this is these polymorphic properties that we were looking at earlier so what we do is we make sure that instead of just assigning IDs we assign these objects which contain both a type and an ID and passing that through Embedata allows polymorphism to magically happen I mean basically if you go into the Embedata source code and search for polymorphic you'll find where that gets gets treated I think there's probably only about one or two occurrences of the word polymorphic or polymorphism in the entire source code for it unfortunately not really documented but it was kind of a last ditch thing to search for polymorphic in there we found that you can just pass in this object that was a good day so yeah that pretty much covers it so basically all we want to show is that Embedata is not necessarily as rubbish or limited as a lot of people seem to whine about on the internet if you actually have a look at what it can do there's a huge number of methods that you can override a huge number of properties that you can tweak on the built in adapters, serializers this kind of thing and even if that isn't quite enough for you just having an understanding of the structure of this code means that you can just make your own little tweaks to it, little adjustments as you need to just to be able to handle whatever horrors some API designer might be able to throw your way in 15 years time in total six months we've understood it really for about four or five weeks we've done a lot of rewrites in the last four or five weeks but those rewrites are things that took us a month to two months originally and in a day we brought in all this stuff and went oh actually we can get rid of 90% of the code it's all already there the problem is it's not documented you don't know it's there it was much much easier all the little nasty little bugs that meant it sort of worked have gone I don't see we've actually got any Embedata bugs we were using beta 3 we haven't used beta 4 yet we've used beta 3 and it just works that's the other gotcha if you're looking at stuff about Embedata if you're looking at stuff that's a year or so old or older just discard it if it's less than six months old it's probably okay if it mentions beta 3 then you probably ought to be believing it if it doesn't, leave it alone no we're only adding what 15 lines of code um we're making the same query to the server which is the slow bit that Ember would be working well a little bit but we're not talking about megabytes of data I'm sure if you were something has to transform it so it has to be transformed at some point into the store we may as well make it easy for the store and do the hard bit under our control so one final thing if you like what we do we will be hiring in the autumn £30 at the moment in equity on Cedars crowdfunding site so if you've got £30.46 in your pocket and you'd like a share of Silver Curve you can go to Cedars and buy one we're taking investments up to £100,000 each so if any of you have got £100,000 in your pocket wearing a hole in it, come and give it to me any questions other than the obvious ones let's just hear it for these guys between the adapter and the serializers you talked about the situation in which you had cases where I record in Ember data may translate into multiple stores multiple records in the back end whose responsibility is that between the serializer obviously needs to transform into the format that the server is going to need it seemed to make sense if you've got, say you've got three calls to the server to set the data to have the adapter so the serializer return an object with three sub-objects in it which are the three parts that we need to send back what we're trying to do is say the adapter doesn't do any transformation there are some points where that's not possible because there are some transfers we need to do before you make the second call to the server but if we take the target that it should do any transformations and if we find it has, think carefully about it then that seemed to work very well for us equally the serializer is synchronous so it can't talk to the server so all the serializer is allowed to do is transform data so you do have the option in the adapter to even take the output from the serializer and mess it up with it again we decided not to take that route and just say whatever comes out of the serializer should be exactly as open data wants it and that embedded rest serializer was a great help in that using the metadata of the model to navigate and find out is this an object or an array or a property what should I do with it so the little option property about polymorphism how much code is that impacting with the member data very little really it basically just uses it to say when you're pushing an object into the store what type it should be so rather than you create a record of a specific type and that type gets passed down all the way through to your push object for example it just pulls it out from there so it's just kind of a I think it's like a three case thing of is it a number is it a string is it an object yes okay we must be polymorphic and it just handles that you see that here where the type tells it which array to get it from in your side loaded data so it's going to do a pluralisation of that find the template properties on that block obviously if it knew the type in advance it would know exactly where to go and you'll see a few places in the back end where you have a all through the store you have methods that take store type and id and what you see in polymorphism is it overrides that type so the type is the base type but then override it with a descendant type once it works it out and you can actually make use of that in your adapter for example if we're going to go and find a number but the server sends back a cat I need to store it in the store as a cat not an animal so in the adapter we've actually changed the type that we send back to be a cat rather than animal and we do that by rather than having a list of animals have a list of cats, a list of dogs whatever else unfortunately zero documentation on that you have to actually go and go through the exceptions in the code and find out why ember data doesn't like it it's not fun who's using data driven back ends that they didn't write in this room wow lucky people any vacancies go so some of you can sleep at night it's really not fair ok thank you very much