 So, hi arferwant. I'm Duncan. This is my first time giving a technical talk in about 10 years, literally. So a couple of slides please. I've been playing around with meat here quite a lot, and I found that every time I understood particular topics and features of the main gluten wall depth, it actually vastly improved what I was able to deliver and led to a bit of refactoring. I want to cover a couple of those topics on that. One is around the use of the transform function and the other is around EJSOM, which I kind of regard as being fairly similar to each other, trying to explain that. Because it's the first time in about ten years as well, I'm not going to do any kind of live coding stuff, but I've got a bunch of stuff. Here's one I've made earlier. What I'll do is I'll try to walk you through and show how the concepts have been applied through a very, very simplistic example, which is kind of derived from something that I did that was far more complicated. So, really, it's around being able to reuse your code and basically create a model that is shared between the back end and the front end. So, I guess kind of MVC is a common term, whether you call it a framework or a principle, that's used in development. Meteor does really have those kind of capabilities, but it's maybe not that wonderful at showing off those capabilities. So, just so I know kind of know what sort of speed to go at, can you please raise your hands if you use the transform function on a collection in Meteor before? Like, seriously, the more people that raise their hands to it quicker, I'm going to go through it. So, if you have, then I don't want to have to regurgitate something you already know. OK, so I'm going with a handful. So, I'm going to spend a little bit of time on that, then, and show how it works. So, you know, again, I may get some of the terminology wrong, I'm a little bit out of practice on this, but generally speaking, what the transform function allows you to do is take your regular old objects that are coming out of your Mongo environment and enroute, model them more as proper objects. So, I've got a couple of links and I'm going to, I've already stuck this on SlideShare, so you can have a look at the links. You know, it's in the official documents and then there's another link there for, you know, quite a kind of good step-by-step example. And the way you typically use it is you define the transform function on your collection, typically creating a new object. You take whatever's come out of your Mongo document and you use that to extend the object, basically add on all the properties. And then once you've done that, you can then basically further extend the prototype of that object in JavaScript with additional properties more likely to be functions or methods, whichever your preferred partners is. Now, that's all the kind of theoretical stuff, so what does it really look like in practice? So, can everyone see? Do I need to make it even bigger? We don't have like a proper projector, so those at the back, hands up, can read. Awesome. So there, this is just to say, just so you know all the packages, I took away the Insufio package but I've left the Auto Publish package just so that kind of everything gets to run to the client. So then, let me, it's almost okay, so all I've got is, this is my base template, I'm inserting another template called play. And so, what I've done, if I, so the basic premise here is, I've got kind of a hierarchy that I'm going to be going through, so my base collection is a game, I chose something that everyone's kind of familiar with, is a game. Within a game, you have any number of teams, and within the teams you have any number of players, and a game can't start until you have at least two teams and at least one player in each team. Okay, fairly simple, straightforward kind of concept. All right, so if I click the create game button, that basically has just a pretty simple, it's called the method on the back end to create a new game, which all it does is say, insert a very simple document at this stage, right? So there's no teams, it's basically completely empty. All right, now, this is where it probably starts to get a little bit more interesting. So you see that the, although all I've done is insert the document, I've actually got more information, obviously I've got the ID that's also generated when I stuck it into the database. But also I know how many teams are on, how many players there are, whether or not I'm ready. So how's that actually coming about? Well, so this is where Meteor's Magic really starts to come in. So firstly, within the table that I'm just declared, so I've said for each games, so games is a very simple handler, just says, find all the games. Okay, great, so all I'm doing is pulling out whatever games that are in the database through this. So nothing, it shouldn't have any kind of funny stuff going on. But by doing that, now I've got a teams, but I can actually, because teams is an array, directly within spacebars through Meteor's Blaze component, I'm able to chain that property, so being an array it has a length property, so I'm directly displaying the property. And that's what gives me this value here. So simplest form of modelling, right? No, I've not actually used anything funky at this stage, but I've been able to recognise that within my each, so my each is for all the games. My calling games I've basically set data context within this block will be whichever game is being looped through. So it could do this.teams.length, but it won't make any difference, it automatically infers what this is within the current data context. So I can not only represent one property, which would show it as an array, but I can actually use chaining to pull out the length property. All right. So then, that's all well and good. Then I've also got this thing called player count, which is also telling me I've got zero players in my game, no surprise there. But if I look at where is that coming from, you can see actually that's not included in my helpers. My helpers are game-selected, game-selected teams. So I've got player count, but player count wasn't if I go back to my insert. It wasn't something I inserted in the database, it's not a property. So where on earth does that property come from? And the answer is that that's where the transform function comes in. So normally when you declare a collection, it looks something like this, right? Everyone custom to do that. You're also able to pass it a function, and that function is a transform. You can pass it several things actually, but a transform is one of the functions you can define on a collection. And that takes a document as its input to the function, and then you can do whatever you want with it. So basically for every document that is pulled out of MongoDB, it will run it through this transform function. And instead of returning the document when you say games.find, it actually returns whatever the result of the transform function is. So in this instance I've told it to return a new game. So what's that? How does that make any sense? So this is where I've stuck things in the model folder here. So this is where it gets, I think Meteor gets really pretty awesome. So my model folder is something that I'm sharing between my back end and my front end. This is where the whole isomorphic JavaScript comes into effect. So my constructor, as I mentioned, just said to take the document and then when I'm creating the object, just use the underscore library. The underscore is completely integrated in Meteor as one of the underscores in jQuery. They can't guarantee it will work and just extend it with whatever the document had. So that means I know when I put the document in, I had start and team. So my games object should have those two properties. But it still hasn't got this player count. So where's that coming in? So now I can also extend the prototype of that object as well with whatever I want. So at the point of pulling a record out of the database, I can basically go from being a dumb, effectively, JSON string and now make it into a proper object complete with additional properties and methods and everything else I want to add. And that's basically what I did. So I've created a function. I've called it player count. It extends the game object. By doing so, I've basically modelled this on the back end. Sorry, I'm more of a back end of the front end guy if you haven't worked it out. I've modelled this on the back end. It comes out of my database when it's been called for. It runs it through a transform function, turns it into an object and then it's available on the front end. So on my front end, I refer directly in spacebars to that function. Spacebars will basically can handle it with you. Here, for example, this is actually a property. Here is a function. It doesn't matter which. You don't need the brackets afterwards to execute the function. It will do it automatically. You can chain props and or functions or properties and functions. So if player count returns an array, I can do playerCount.length and it would know to execute it and then return the link property. So the really important thing is here is what I've not done which is what I naively did when I first started using Meteor. I've not defined this as a template helper. So I've not told my front end to go examine the object to run some kind of function. Rather I've defined it once in more of my back end in my model environment and now wherever I want anywhere in my front end I can call playerCount up because it's also not a template helper. I can also call it on my back end. I can call it in my methods. So it's now available to me anywhere. I want to read the data that's coming out of the database. I've basically taken Mongo, which is a document store and I've basically turned it into an object, sort of an object model database. So then just some sort of things around this. This, what you extend the object you return, sorry, is it's not reactive. So anything that's going to drive Meteor's reactivity is going to be a change in the database. It's not going to be a change in a sort of a calculated property. So if I decided to extend my game prototype with some properties that I've calculated at point of extraction, if for whatever reason that calculation changed without the data in the database changing, that calculation would not be rerun. So that's a very important point, which means that generally speaking, you're not going to be extending your prototype with properties, unless those properties are directly derived from the data and those properties will, and any change in the data will cause that property to change, because then you change the data, the reactivity will kick in, a new record will be driven through, it will go through the transform again, new object and pushback. But if that property can be changed through other means, that's not going to drive any reactivity. What it means though is it does make it very, very good for adding on the methods, adding on the functions that are used to calculate or infer things from existing properties within the database. So it's really, really good for that, as in this, so this kind of player count. So that's really where the transform function comes in, so it allows me to determine that and run a function that gives me this. It also tells me that I'm not ready to play yet because I don't have at least two teams and I don't have and every team is not ready. Now that's the kind of second part of the talk I'm about to jump on to. So at this point you can see I've also taken, sorry if you're not big on functional programming and underscores and all that sort of stuff. Basically what this says is I've reduced the number of players to a single value by going through all of the teams and taking the size of the player's property from that team. So remember we've got a game team players. At the moment there aren't any players, so by definition that's not going to return me anything meaningful. Okay, so that's the transform side of things. So basically you've got this really cool way of pulling when something reactively gets pulled out of the MongoDB back-end you've now turned it into a proper object and you're able to use it as an object in your back-end and in your front-end. That's where that isomorphic code really comes in. It's from an MVC perspective. That's the bit where people who are more front-end developers should be saying, hang on, this is like something that is known for the object. This is common. This should be pushed into my model and delivered on the back-end. I shouldn't be continually redeveloping this on the front-end. You can have helpers that span all templates rather than being template specific, but even better is if you don't need to do that, you push everything into the model environment, especially things if you find yourself reusing things on the front-end, basically redoing the same calculations on an object or on a document. Probably the best thing is to use the transform function, basically attach that to the document, then you can just call it directly. The whole point is that you're not repeating yourself, you're defining it once and then you're reusing it everywhere. These are things that are very model-centric. If you've got a calculator value, that's really something that is inherent in the data, but it doesn't have anything to do with defining how that view is displayed to the end user. That's the first part. It's a transform. I'll pause at this point where I jump into a JSON, which I see as the big brother to a transform function. Any questions at this stage? I really hope I'm explaining myself because it's a tricky topic to cover. Also, it's part of the standard functionality. It's in the Meteor document. You can find all about it. Put super simply, JSON pretty much does the same thing as a transform, or that's how it's set up. That's how I see it. Rather than acting at the level of the document, I see it as acting at the level of a property. For example, if in my nice little games environment, if I had one game for the game of Go and another game for the game of Chess, I'd probably got teams in both those game documents, and I might well want to define the teams in exactly the same way. Using the JSON, basically I can do that. I can say this is our teams object, behaves with respect to the database, so I can then get a level of reusability out of that, particularly with respect to some of its methods and some of its functions. It's not at the document level. Transform is really good at doing it at the document level. JSON is really good at doing these things at the level below the document when you're really looking at the property. But it's a bit more complicated to do. I prefer to then jump back. Here's an example of where I've actually used it. To determine whether the game is ready and whether all the teams are ready, I'm looking at to have at least two teams, and then I'll also also go this thing here. The basic saying is every team ready, but that's not a property. That's not like a length count. I'm actually just called a function or a method on a team that it doesn't look like I actually ever defined. But I did. So at this point, I think I can jump into my monger, I think. Remember I've only created a very simplistic game when I inserted it into the back end, right? So this is all at hand. So that's all you can see at the back end of this stage. What I'm going to do now is select this. By the way, for the front end, I'm using the materialize package, which basically gives you all the front end for Google's material design stuff. It's really nice to recommend that. Because basically I didn't have to code anything on the front end because I'm probably shouldn't have that. So then I can edit team. So now I've added the team nitrous as we know there's no blares in there yet. Let me do the back end. So when I've added a team, again, all I've done is I've updated my games object for a game that we already had selected. I'm setting the teams equivalent to the game.teams. So all I did with that is I created a new team. I gave it a name. I pushed it onto my original empty teams array. And then I updated the database. So just to kind of prove on that, that's the kind of constructor as it were for the team. Well it fixes a name and then it's got now created an empty array of players because I didn't tell any players to push in. But then when I look at what's been pushed into Mongo... Sorry, it doesn't mean that we don't. Let me scroll off the bottom. So when you look at what's in the database, this is the game's document. It's not installed as teams. But rather than storing it just like this as you might expect, you know, so you can see there's an array. But within that array I don't just have a standard serialized JSON object with a name and players. I've got these funny kind of EJSON things that have come in. I'm going, okay, so what's going on there? And that's basically the naming convention that Mita was using in the interaction with Mongo when it pushes it in to say, I'm not pushing in a normal property. I'm pushing in an EJSON property. Okay, I have shown you how it knows that's an EJSON property. I just want to know that today. But that's how it kind of looks. That's how it ends up in the database you end up with this funny kind of naming convention. I mean it's not that hard to work out exactly what's gone out. So again, what happened to be able to do that? I need to have these two functions as available on the object that I've pushed in. So remember, I actually created it as a new object here. So because it's an object it's got those functions at the point that I pushed it into the array and at the point I sent it into the database. So because it's got those functions Mita will basically look at those and go, aha, those special predefined functions are there. That means it's an EJSON object I need to do something special with it. So they're not that complicated honestly. So the easiest one is the type name. Okay, so what's my unique name for this EJSON object? So this needs to basically be a unique name. You're defining a custom type at this point. So there's a custom type. You obviously can't use standard names that would just be silly. And you also want to make sure there's no kind of namespace conflicts and things like that. And now you can see it's actually ended up there. That's what it's called there when it's pushed into the database. So that tells me to... Okay, this funny stuff in here I actually know what this is. This is a specially custom-defined EJSON type called Custom Team. I know how to handle it when I'm pushing it in I know how to handle it when I'm pulling it out. Or we actually tell me to or how to handle it when I'm pushing it and how to handle it when I'm pulling it out. And that's the purpose of this. So the first thing we tell it is okay, this is how to serialise it. This is how to take the object and turn it into something that's just JSONable that Meteor can then handle and stick in Mongo. So normally it would be pretty simple. Maybe I should hide this. I'm going to come back to that in a second actually. So that's basically... I want to show you the player object first and then show how that builds up into the team. So that's how to push it in and then you also don't have to tell it how to get it out. So that's where you use this special EJSON object. You say okay, Meteor. I'm adding in my own special EJSON type for you to be aware of. I'm calling it Custom Team and I'm giving you a function and that function tells you what to do once you pull it out of Mongo. Okay? So what I've done to try to take it to another level as well I've done it not only the team but also the player. So if I add into my team I've now put a player in. Let's do that. So now I've got two players in my team. You see it rolls up here so that reactivity is happening. So remember this means I push it into the database and it's pulling it out. It's run that transform function. It's added the function onto it and I've just called that in the front end. So that reactivity is happening nicely. I'm not having to calculate anything on the front end and the model's done that all for me. I'll show you how that looks in the database now. So now I've got because I define custom player as an ej som type as well. So I've got ej som within ej som. So I've got ej som within ej som within a transform. So I can reuse my player object anywhere as well. It doesn't have to be within a team. And I can reuse my team and players anywhere I want as well. So if I then look at players players constructed pretty similar so again, if a type is custom player so this is where I want to start. So the function you have to turn it into ej som to serialise it is basically pretty standard. That's how you normally write a bunch of ej som. And then when you add the type you basically tell it okay. So you'll be past the value so effectively that's the property. The value of the property is what the meter has pulled out. And it will look like whatever you put in. So all the squiggly bits they kind of go away. It kind of uses that as its own naming convention to store it so that it knows when it pulls it out what it's going to look like. So this will basically match the value that you've caused. So therefore you can basically re-insatiate it doing whatever you've done. But just as we did with the transform function I've also extended it here with a method that's very simplistic. I'm not trying to do anything too complicated that defines how to represent someone's name. Some people like to have their family name first some people like to have it lost. I didn't want to implement a method for Indian names because they're different by every state. So yeah, so that's basically added it on there and that's why when you see here this representation is happening here but if I look at again my HTML for that for each player it's just called getFullName that's not a helper I'm just calling the method on the JSON object. So if you look at everything I've done all I do is these are just a selection thing for when I'm adding players that's just pulling back a session variable that's not doing any calculation here The only thing I'm really doing is saying just give me the data but when I'm saying give me the data I'm saying give me all the functions give me my model give me all the JSON stuff all at the same time So the bit that I didn't show earlier then is when it comes to the team So because you can have eJSON within eJSON you need to take that into account as well it's kind of tripped me off first when serialising and deserialising it So in this instance for each of the players because I know that's an eJSON object I basically said so players instead of just being doing the four just same players players I actually have to say I need to then take my player turn that into an eJSON type before I wrap it up in a high-level eJSON and then again when I want to pull it out So my name that's just straightforward I pulled that out exactly the same way I did before but my players I need to run each of my players through how to get it from eJSON back into an object So that So it's a little So what's actually going on then is I'm pulling out my document on the way out of Mongo meteor is basically intercepting it and saying I can see you've got some extended eJSON some eJSON here So before I do anything I need to then run that eJSON function So the first thing it will pick up will actually be at the team level and it will then say you've got players this is the value for players so that's where I'm taking the players but I know because I defined it that actually players is itself an eJSON object so I then need to map it back into an array and call the players eJSON which is going to go through here so it basically runs this it runs it from eJSON for that player to give me my player back and at the same time putting this method onto it so that in my team I've now got an array of all my players but instead of that just being eJSON documents with properties that array I've just constructed is objects it's got methods on it you saw it's got that full name method I can call like that I intend to make that happen so I've done that and then having done that when I return it as the team the team itself has also had like whether it's ready or not which is basically do I have at least one player in my team so that's had that method added on it and then when it comes so when it comes to the game you're like okay so how's it how's it got there from the game and the answer is well because the form function has actually put it into my game as well because even before meteors pulled the transform on the object is pulled out of Mongo so it's pulled it out of Mongo and it's already at that point run the eJSON to turn those properties into more fully fledged objects so from a collection perspective when I'm running the transform I'm not actually passing it a typical document but I'm actually passing it an object that's got properties and now methods as well because that's coming through eJSON so when I create the new game and I extend it so extend will add to the new object all of the properties of the objects that it's coming from and at that stage eJSON's already run so it's also adding all of the methods so I basically end up with this completely fully fledged object that are at several levels of nesting that I'm then throwing up to the front end so that I can inspect properties I can run methods at the game level I can throw it on a table I can look at properties within objects of objects and then even down here I've basically got an object within an object and I'm calling a function on it because I'm running an each and each and each and me too natively understands the data context each time of what I'm pushing so really eJSON and the transform function are incredibly powerful it allows you to build this whole model and that's where the whole kind of isomorphic stuff really comes into play it's talked about a lot with me too and I didn't really understand what would the point be because you do stuff in the back end you do stuff in the front end you might be using the same language but you're not really reusing much of it the transform in eJSON is where the reuse really really starts to come into play so any of the stuff on those objects or methods I can call them on the back end I can call them on the front end they're all defined in one place I just stick them in for me personally my models folder I'm not sure how the best way to structure a meteor project is I think that's the topic of ongoing debate but it's then available across the whole stack for me to use as you've seen it pretty simply so just to prove that it's ready two players and one and one and the other I know it's ready to go so that's eJSON, that's transform in summary transform is fantastic at the document level one thing that's objects eJSON allows you to make proper models out of your properties that's one of the reasons why I know just how awesome in meteor is and why the morphic code actually makes sense as opposed to just happening to have the same language on the front end and the back end and that's me any questions? you mentioned earlier that most of the transformation are used in the front end of the object transformations were used in the front end the bit I don't know is whether the transform is cos I don't know meteor so I don't know if it's sending an object over the wire cos I think meteor says it's data over the wire so I presume that having that isomorphism means that actually the transform is applied on the front end when it pulls it out of mini-monger cos mini-monger is already ready for the back end you can check that in the java 6 console so you can go to your site and you can just run find command on the mini-monger the client side database and see if it's actually storing the object model I'd actually interrogate the mini-monger itself on the console in the java 6 console I mean in fact you can try it now I've never tried interrogating it tonight what's the... it's a collection name and it's not defined yes so they kill the... but defined you already know that it's been turned into the object by the time you pull it out from a front end perspective yes you're dealing with it as an object what I'm saying is I'm not sure where where in the process it's all happening so basically my understanding with meteor collection it's both defined on the server and on the client so the transform function if meteor receives the data from the mongo it runs the transform function in the server I don't know so I don't know if it's running I don't know if it's running the transform on the server before it passes the data to the mini-mongo or if it's just passing the data to the mini-mongo and the front end is running the transform as it pulls it out to the mini-mongo to objectify it for the client I suspect that's what's happening but I don't know because it's isomorphic so although you've written the model once meteor's made it available to both the front end and the back end so when you call games.find on the front end it pulls it out of the mini-mongo if you're calling it within a method but how would it send an object from the server to the client that's why I assume it must be doing it on the mini-mongo if I was meteor it would be just sending the mongo JSON data to the to the client because that would be more faster because this objectification happens out of the mini-mongo into the client I'm assuming but I may be wrong because meteor's using DDP it's using its own protocol it's not using standard joseph your server side aren't you using your new methods so how would that happen so the whole thing with meteor's is awesome even at the simplest level when you define the collection even without a transform normally you've defined you've defined them in one place but actually that's available on both the front end and the back end even if you define it without a transform so my point is that if you have that running on both back end and front end and you only need it on the front end why do you need to transform the data in the back end so right so why not just transform the data when you have the decent data on the front end which means if you really wanted to meteor does allow you to differentiate whether it's meteor as client or a server so in theory you could do that within the transform you could do this much on the client do this much on the back end if you really wanted to but part of the point if I call in console log inside a console you can see where it appears where console log inside is meteor like the prototype function where you're defining or in fact when you're extending so my assumption is it's actually running on both it's isomorphic so when it pulls it out a MongoDB on the back end when it pulls it out a mini Mongo on the front end it will also run it through the transform again the whole point with some of that isomorphic stuff is for example on the front end I can say is the game ready so if on the front end I click game start and it makes that method called to the back end to start the back end can do a sanity check to make sure that someone's not filled with the front end and also call exactly the same method on the same logic and before I start this game is it actually ready and although it's all intent but it almost looks like it's calling on the same object it's not calling on the same object it's what you would have actually done is just taking the game ID you've gone to Mongo and said okay let me pull that game out it's run through the transform function again so that means that method that method is called is available to it now let me check it again to my kind of back end verify and insert and copy of the object that hasn't been filled with by the client so again that's where some of that isomorphism comes into play you can basically utilise exactly the same methods functions whether they be sanitisation checks or whether they actually do stuff on the front end and the back end on the back end they're running off of the back end data stuff and on the front end they're running off of the mini Mongo that's what the filter is about with all the methods on the front end you know that the ones on the back end are still safe and sound the ones you define which is really cool but hard to get your head around sorry I can't hear that's a good point when I've tried to apply some of the like a filter to a publication I've got into difficulty when I've tried to do that on whatever properties I've made eJSONable and I'm not sure whether that's a plug or a feature because in theory you know meet or should be intersecting that and say that I know you've defined this as eJSONS and you kind of convert the query to search for that as an eJSON object but from what I see I looked at that about a month ago and it didn't seem to be working maybe it may have been addressed by now maybe just my bad code because yes certainly you wouldn't want to have to define a query to utilise the eJSON naming convention because you wouldn't expect any guarantee that that naming convention would be consistent between releases that should be the reason why you should not convert on the server side and you only convert the object when using it in the front end but the eJSON affects how it's actually stored so it's been stored within the Mongo using that naming convention so how much of this was I think I'm stressing on what's purely incidental to this but so as far as this goes was this data structure that you've got that was really just for the illustration on what we do actually recommend doing data structures like this in documents because I think that's the way I see eJSON is it makes most sense to to use eJSON when you've got a component of your model that is being reused across different collections so Mongo Mongo is not a relational data store right so but if you think of it in terms of RDVMS terms if you've got a kind of a common definition that would need to be shared across multiple tables in Mongo the same thing can happen you might have a common definition that needs to be shared across multiple collections so each document basically refers to the same master copy that's really where I think eJSON comes into its own and has its strengths because it allows you to do just that in this particular example if I if I'm only using it within a single collection and that's why I use the example like what if I have a game of chess what if I have a game of goat if I'm only using it within a single collection I can pretty much do everything I just did with eJSON through using a transform function but then I'm not storing it I don't have all this funky eJSON stuff in my backend so there's nothing to stop me running like as I pulled the data out just running a map function on the array and objectifying it that way before I put it in so all of that can be done using the transform function so eJSON is really best used I believe the meeting or team may tell me a problem if you have a particular custom data type that you want to reuse across multiple collections I'm not sure if I forgot maybe if you've got a say you've got a collection of sales documents and you've got a collection of purchase documents but you know that there are some people who you buy from and you also sell to the same people so within each document you might want to define a business partner and you want to make sure that business partner is exactly the same in your sales orders as it is in your purchase orders so you can define business partner as an eJSON object so that when it gets pulled out from the sales orders it's got exactly the same methods and setup as if it had been pulled out of purchase orders so by doing that you've defined that eJSON once you've not repeated yourself you've not redefined the transformation process twice within the same set of documents or it's effectively the same object it's one of those ones where I think the more complex your models get the more the more useful it becomes if you're dealing with fairly simplistic single collections where or even a set of collections where they're completely segregated there's nothing common between them you probably don't need to use eJSON because you can just take your transform extend, extend, extend extend as much as you want two properties, two sub-properties et cetera Any last questions? Or are you even going to argue with me later about any other stuff that's going on Like I said, first time I've done a technical talking about a decade so hopefully we'll continue